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ABSTRACT 

This report reviews the progress made by the New York 
State Education Department and local school districts in developing 
and implementing a comprehensive student evaluation program for 
elementary and secondary education, with the assistance of local 
school district personnel and technical consultants, the department 
concentrated its efforts on identifying the most pressing information 
needs and the characteristics of instruments adequately supportive of 
the decisionmaking tasks of today and tomorrow. The "interim" program 
first described in the report makes the best and the most systematic 
uses of existing methods and information to evaluate current 
cat^^gorical and general educational programs, and gives the best 
answers currently possible to pressing, performance-related queiLions 
being asked by the educational community. A "projected" program, 
described in the final part of the report, provides an examination of 
two state*of-the-art problems that must be overcome if the 
perf ormance^related questions asked by educators, the State 
government, or the public at large are to be answered. (Page 140 may 
reproduce poorly.) (Author/JF) 



ERLC 



EVALUATION 
ELEMENTARY AND 
SECONDARY 
EDUCATION 



Ui OE P-i^P T V'l-N T OP MEAL TH 

NATIONAL INSTITUTE OF 
f DUC AT 10^ 

.... \ :.\ I f)SOk<^3iN 

, ■' 1 - s ■ \t ■ i A kiL Y ffEPKf, 

- .'.S'lTgU Of 



A Report to the Governor and 

Members of the Legislature 
Prepared Pursuant to 

Section 3602 of 
the Education Law 



o 

Q The University of the State of New York 

THE STATE EDUCATION DEPARTMENT 
^ Albany, New York 12224 

^ March 1974 



THE UNIVERSITY OF THE STATE OF NEW YORK 



Regents of the University (with years when terms expire) 

1984 Joseph W. McGovern, A. B., J. D. , L. H. D. , LL. D. , D. C. L. , 

Chancellor- - New York 

1985 Everett J. Penny, B. C.S., D.C.S., 

Vice Chancellor White Plains 

1978 Alexander J. Allan, Jr., I^L- D. , Litt. D, Troy 

1972 Carl H. Pforsheimer, Jr., A. B. , M. B.A. , D.C.S., H. K. D.- Purchase 

1976 Edward M. M. Warburg, B. S. , L. H.D. - New York 

1977 Joseph T. King, LL. B. Shelter 

Island 

1974 Joseph C. Indelicate, M. D. Brooklyn 

1976 Mrs. Helen B. Power, A. B. , Litt. D. , L. H. D. , LL- D. Rochester 

1979 Francis W. McGinley, B.S., J.D., LL. D. Glens F^lls 

1986 Kenneths. Clark, A. B. , M.S., Ph.D., LL. D. , L. H.D., 

D.Sc. -- Hastings 

on Hudson 

1983 Harold E. Newcomb, B. A. - Owego 

1981 Theodore M. Black, A. B. , Litt. D Sands Point 

1988 Willard A. Genrich, LL. B. , L. H. D. Buffalo 

1982 Emlyn I. Griffith, A. B. , J. D. Rome 

Presi d ent of the University and Commissioner of Education 
Ewald B. Nyquist 

Executive Deputy Commissioner of Education 
Gordon M. Ambach 



ERIC 



THE UNIVERSITY OF THE STATE OF NEW YORK 

THC STATC CDUCATION DtPARTMCNT 

Orricc or the PACSfocNr or thc t/KcvcRSirr 

A^<D COMHiSSiaNCA or COUCAttON 
ALBANY, HEW YORK 12224 



To the Governor and the Members of the Legislature 
of New York State 

At the 1973 Legislative Session you amended Section 3602 of the Education 
Law to include the following provision for elementary and secondary education 
evaluation: 

On or before February first, nineteen hundred seventy four, 
the commissioner shall report to the legislature on the manner 
in which the funds provided under this subdivision were spent, 
including but not limited to the names of the school districts 
which received funds and the amount, the disposition of funds 
to other agencies or by the department, the programs which 
were financed from the funds, and an evaluation of such 
programs and their relationship to programs funded with 
moneys provided by the federal government under the provisions 
of the Elementary and Secondary Education Act of nineteen hun- 
dred sixty-five and programs funded with moneys provided by 
special aid programs of the state of New York, and the extent 
to which such programs have been incorporated into the regular 
school curriculum and recommendations for future action* An 
amount equal to two per centum of the total appropriation for 
categorical urban aid shall be made available to the department 
of education from such total appropriation for the evaluation of 
such program and for the development and implementation of a 
comprehensive student evaluation program for elementary and 
secondary education.^' 

I am pleased to submit a first report to you on the status and results 
of the evaluation to date. 



I urge your review of the '^interim" and '^projected" programs described 
in this report. Your continued support of this statewide evaluation program 
is requested. 



Respectfully f^ubmitted. 




Comnus^^ionc r of Educa 
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I. INTRODUCTION 

New York State government officials^ from the executive and legislative 
branches have engaged in increasingly frank and open discussions of the 
problems associated with supporting local school district efforts to meet the 
educational needs of the citizenry. This constructive dialogue has sometimes 
been incorporated into printed reports, such as the study on ^'Urban Education 
Evaluation Reports for the Legislature" in 1978, and the "Preliminary Proposal 
for 1974 Legislation on State Aid for Elementary and Secondary Schools" this 
year. This report to the Governor and Legislature on the Evaluation of Ele- 
mentary and Secondary Education is expected to foster further mutual coopera- 
tion among all parties interested in high quality education at the lowest reason- 
able cost* The Report sets forth the Department's conception of how best to 
proceed to insure this objective, along with the actions taken during the past 
year consis tent with this conception, and it invites the people and their elected 
representatives to review the Department's direction and the support it merits. 

This report reviews the progress the New York Sti.te Education Depart- 
ment and local school districts have made in developing and implementing a 
Comprehensive Student Evaluation Program for elementary and secondary 
education. On May 17, 1973 the Governor and Legislature amended New York 
State's Education Law to provide funds for the program developed by the De- 
partment (with advice from local districts) over the past three years. The 
report describes the work to date and indicates the plans to continue and 
expand these efforts during the fiscal year 1974-75. 
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In this introduction, three questions being raised both within and outside 
the educational community are identified. Because of two national '^state of 
the art" problem areas which present barriers to answering these questions, 
the Department has developed both a long-term or "projected" program which 
addresses itself to overcoming the ^'state of the art" problems, and a near- 
term or "interim" program which provides the best possible answers given 
the existing^ constraints. 

The remainder of the introduction provides an overview of the "interim" 
and "projected" programs and summarizes the findings and the progress that 
has been rpade to date within both programs. A complete discussion of the 
"interim" program is presented in Chapter 11. . A similar detailed discussion 
of the "projected" program is presented in Chapter IIL 

A. QUESTIONS BEING ASKED OF 
THE EDUCATIONAL SYSTEM 

The intent of all the work activities presently in progress or being 
planned with the funds presently allocated is to answer three performance- 
related questions being raised both within and outside of the educational 
community. The questions are: i- ^ 

A. To what extent are New York State's elementary and 

secondary students successfully mastering basic skills? 



The questions listed are very similar to those in the legislative 
Commission on Expenditure Review in their report- -Program Audit 7. 1. 7i, 
June 30, 1 972 —entitled Urban Education Evaluation Reports for the Legis- 
lature fthe reader of this report might be interested in reviewing the Educa- 
tion Department's response, pages 19-29 of that document). 

ERIC 



13. What types of instructional programs (or projects in the 
case of categorical aid programs) are most effective for 
students of different characteristics? 

Ci To what extent are the best instructional planning, 
management and evaluation practices being used at 
different levels of the educational system? 

It is generally believed that answering these three questions in a fair 
and meaningful way for all school districts in New York State is presently 
feasible. The Department, as a result of analyses over the past three years, 
is convinced that it is "developing a program, which is described in Chapter III^ 
that will adequately answer the questions. However, the Department is also 
convinced that an ^'interim'* program must be used until the necessary devel- 
opmental and pilot work is completed which overcomes the ^'state of the art^' 
problems identified in Chapter III. 

B. AN OVERVIEW OF THE REPORT 

The Department has concentrated its efforts during the past year in 
identifying, with the assistance of local school district personnel and technical 
consultants, the most pressing information needs and the characteristics of 
instruments (tests, surveys and the like) adequately supportive of the decision 
making tasks of today and tomorrow. This report describes the achievements 
already realized, and the ways in which these accomplishments are intended 
to serve as building blocks for even more significant progress in the future. 

The ^'interim'^ program described in Chapter IF makes the best possible 
uses of existing methods and the most systematic use of existing information 
to evaluate current categorical and general educational programs, and gives 
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the best answers currently possible to the three pressing, performance- 
related questions being asked from within the educational system and from the 
Governor, the Legislature, and the public at large. Conclusions in Chapter II 
have to be viewed with caution and interpreted carefully because of the exisM ig 
"state of the art^' barriers discussed in the beginning of Chapter III, 

The "projected'^ program- -A Comprehensive Student Evaluation Program- 
is described in Chapter III. Chapter III provides an examination of two "state 
of the art^' problems which must be overcome if the three questions described 
above are to be answered in the future. The description of the "projected^' 
program indicates how, as the various components are incrementally imple- 
mented statewide, the existing "state of the art" barriers can be overcome 
by: (1) improving testing methodologies so children are tested for specified 
and more useful purposes; (2) providing better information to educational 
decision makers and the public; and (3) improving instructional planning 
and decision-making structures, 

C. SUMMARY OF FINDINGS AND PROGRESS — 

THE ^'INTERIM" AND "PROJECTED" PROGRAMS 

Since many readers will not have the time to review this entire report, 
a brief summary of the findings and progress made in developing and imple- 
menting both the interim and projected programs is presented in this intro- 
duction. This summary is organized in a manner which begins with either 
the "interim" or "projected" program and then reviews the various work 
activities under each of the three performance -related questions already 
outlined under "A" above. 

ERIC 
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i. Description of Progress Toward the Interim Program (Cha pter II) 

a. Question #1; To what extent are New York Staters el#^rn#^nfary 
and secondary students successfully mastering basic skills? 

1) Study of Local School Dist ric ts^ Nat i onal Standardi zed 
Test Programs ; For many years many individuals 
both within and outside the Department have expressed 
a desire to know more about local school districts' 
evaluation programs especially regarding their use of 
national standardized tests as a means for answering 
question #1 above. Because of this interest, the Depart- 
ment, as part of the 1973-74 interim pr,ogram, designed 
and is implementing a study: a) to obtain information 
on the scope {i.e. extent to which schools admini^ner 
national standardized tests, which tests are nscd, when 
the tests are administered, and what use are tr^uUs 
given in terms of reporting within the districts) of test^' 
being used; b) to determine the extent of the total finan- 
cial commitment to such programs; c) to explore the 
potential of using results on nationally standardized 
tests as a possible alternative to rather than an ad- 
junct to the Pupil Evaluation Program (PEP), and 
d) to determine the achievement levels of pupils in 
New York State schools in comparison with national 
norms. In the data analyzed at this time, answers 
to '^a" and "b'' are presented in this report. Answers 
to the remaining two components of the study will be 
released in May 1974 when a substudy which correlates 
1973 PEP scores with two national standardized test 
norms will be completed. 

The following is a summary of the findings included 
in this report. 

Reports were submitted by 94% of all public schools 
statewide enrolling 95% of all public schools statewide. 
During the 1972-73 school year, 90% of these schools 
administered a reading test in one or more grades, 
testing 75% of their total grade 3-8 enrollment; S(^% 
administered a mathematics test in one or more grades, 
testing 61% of their enrcjUrtifcnt, About 75% of the schools 
using standardized testt. report administering them in 
the spring months as compared with ZS% in the fall. 
Most of the schools use the Metropolitan Achievement 
Tests, the Iowa lests of Hasic Skills, or the Stanford 
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Achievement Tests* Even though most school districts 
did return questionnaires regarding the scope of their 
testing program, there are variations in: a) how much 
testing different types of districts administered: b) time 
of testing; and c) tests used. 

School districts reported spending a total of roughly 
$2.4 million on standardized achievement testing in 
grades K- 12 during 1972-73, an average of about $. 70 
per pupil enrolled. Most school districts report 
summaries of their State standardized test results to 
school board and school staff members and about half 
the districts report State standardized test (PEP) 
results to district residents. Less than half the districts 
report summaries of results on nationally standardized 
tests to their school board and school staff members 
and only 21% report nationally standardized test results 
to district residents. School districts that report teat 
results to district residents use district newsletters, 
public meetings and formal releases to news n.edia with 
about equal frequency, 

2) Comparison of Regents Examination Results in Biology , 
English, French, Geometry and Social Studies with 
Results from Commercially- Published Standardized Tests 
The administration of Regents examinations have a long 
history in New York State. Many educators and students 
have questioned whether they should be continued and 
others have suggested that commercially -published 
standardized tests replace Regents exams since they 
would allow the State to compare its students with 
national norms. The Board of Regents have concluded 
that the examinations should be continued until better 
examinations become available. The Regents believe 
that standards must be set at the State level* It should 
be understood that Regents examinations are generally 
only^taken by students with average and above -average 
scholastic ability. This study examines the feasibility 
of using the commercially-published tests in place of 
Regents examinations and also attempts to answer the 
following two additional questions: 

1. How do the achieven:ienl test scores of 
New York State Regents -level students 
compare with scores obtained by high 
school students nationwide? 



2* How closely currelated arc Regents 
examination and standardized test 
scores in any given subject? 

The rasults o£ the study indicate that the achieve- 
ment of New York State students in June of 1972 was 
generally superior to that of students nationwide in 
English and French and approximately equal to the 
achievement of students nationwide in biology, geometry 
and social studies, The study also concluded that the 
correlations between Regents examinations and stand- 
ardized tests scores range from ,67 to .77. About 
half of the variance of one test is predictable from the 
variance of the other. 

In examining the question as to whether it appears 
advisable to substitute these national standardized tests 
for Regents examinations, the report indicates that after 
analyzing the advantages and disadvantages of both tests 
that the Regents examinations will be retained until 
better tests are available at a reasonable cost. 

3) New York City Prototype Study of Urban Education and 
Elementary and Secondary Education (E. S. E. A. ) - 
Title I Programs - School Year 1973-74 ; As part of 
the total evaluation effort, the Department reviewed 
all the problems associated with relying exclusively 
on local evaluation reports and concluded that it should 
test the feasibility of developing and implementing a 
Department (as opposed to district) managed evaluation 
program. In the work completed to date, pretest scores 
from the California Test Bureau/McGraw-Hill Com- 
prehensive Test of Basic Skills - Form Q, Level I 
(reading comprehension section) in reading for all 
grade five students in a sample of ten New York com- 
munity districts have been analyzed to provide answers 
to the following questions. Both the questions and the 
findings are summarized here. 

The first question is-- Are the ten districts qualified 
to be Title I and Urban Education Districts? The data 
from the pretest indicates that the answer is "yes^' in 
that in a normal population of students taking a stand- 
ardized test one should find that 50 percent of the 
students are scoring at the 50th percentile or below. 
In the sample of all fiftli graders (24,172), 17,403 
students or 72 percent of the students were found to 
be below the 50th percentile. 



The second question addressed is -- Are all qualified 
students in either E.S>E. A, - Title I or Urban Education 
Programs ? The data indicate that t'wo-.thirds of those 
likely to benefit (i. e, those below the 50th percentile) 
from both compensatory programs are not being served 
by either of these programs, The evidence is clear 
that the compensatory funds now available are not 
sufficient to serve all of the students in need if need 
is defined as all students below the 50th percentile. 

The third question addressed is-- Are unqualified 
students in either the Urban Education or E.S>E. A. - 
Title I Programs, and if so is it at the expense of 
qualified students? The answer to the first part of 
this question is '^yes" if unqualified students are defined 
as those having pretest scores at or above the 50th 
percentile. There are 678 fifth-graders in this category. 
If these individuals were replaced by students below the 
50th percentile, an additional 4 percent of the students 
in need could be served, 

The fourth question analyzed is -- Are some student s 
more (qualified than others for entry into E.S.E. A. - 
Title I programs by virtue of their educational deficiencies? 
In defining who might be considered more qualified, the 
Department divided the scores below the 50th percentile 
into two groups. Those scoring in the 1-Z5 percentile 
range were arbitrarily defined as '^most qualified'' and 
those in the 26-50 percentile range as "qualified^'. The 
pretest scores indicate that of the 72 percent of the 
students scoring below the 50th percentile, 42 percent 
scored in the bottom quarter (1-25 percentile range) 
and 30 percent in the second quarter (26-50 percentile 
range). 

The last question addressed is--I s sy stematic 
preference for admittance tji^. S^ E. A^ - Ti tle I or 
Urban Education Programs given to students in the 
"qualified*^ group or the ^'most qu alified^ ' grou p, or do 
preference decisions app e ar to be random? The evidence 
in this report suggests that relatively few students 
(678 students) from the unqualified category (i.e. over 
the 50th percentile) are in the two compensatory programs. 
Clearly these figures indicate lhat there is no concerted 
effort to fill these prngrnms with unqualified students. 
Nor do these numbers suggest that unqualified students 



are even being selected randomly along with other more 
qualified students. It may be human error that some 
unqualified students are provided with compensatory 
services^ or it may be that the costs of perfection in 
targeting compensatory funds are simply prohibitive. 
In terms of comparing the '^qualified" and the "most 
qualified" data, there does not appear to be any clear-cut 
preference pattern between these two groups in either 
compensatory program. Arguments can be made that 
the "moat qualified" group should receive all the ser- 
vices but on the other hand it could also be argued that 
greater improvements could be made with the 26-50 
percentile group. These issues cannot be resolved 
until more data are available. 

4) Evaluation of Urban Education Projects - School Year 
1972-73 ; This section of the report reviews: the 
purposes of the Urban Education Program; 1) how 
projects are initiated, approved, financed, and amended; 
2) a profile of Urban Education projects in terms of 

(a) the degree and form of community involvement in 
planning, (b) whether the projects were regular school 
year, summer only, or twelve month projects, (c) the 
average duration of offerings, (d) enrollment by ethnic 
group and project type, (e) type of participant (i. e, 
public versus nonpublic and expected versus participating), 
(f) staffing patterns in projects, and (g) component 
activity costs; 3) how resources are distributed between 
the Urban Education Program and other categorical aid 
programs and v.ithin programs. 

The last portion of this section of the report pro- 
vides an overview of the methodology used in evaluating 
and results of Urban Education projects. The results 
are clearly encouraging in tliat the sample of projects 
used for the analyses indicates that for all components 
that the actual average monthly gain from the projects 
was . 94 as compared with a predicted gain of .61, 

5) Testing to Measure the Effectiveness of Bilingua l 
Educati on ; The Regents Position Paper on Bilingual 
Education places a high priority on this area of need* 
Since more compensatory funds are now being targeted 
to Bilingual Education programs, the Department has 
committed itself to improving evaluative information 
for these programs* Specifically, efforts are now 



underway: M to measure the degree to which linguistic 
skills taught to students, whose major or only language 
is Spanish; are being successful in equipping the student 
to read, write, understand, and speak in English; 2) to 
develop valid tests which measure student achievement 
in the areas of mathematics, social studies and science 
when instruction is given in Spanish; and 3) to evaluate 
a sample of Bilingual and English as a Second Language 
programs which have a promise of success. 

Question #2: What types of instructional programs (or 
projects in the case of categorical aid programs) are most 
effective for students of different characteristics? 

I) New York City Prototype Study of Urban Education and 
E>S,E.A. - Title I Programs - Scho ol Ye ar 1973-74 ; 
As previously indicated this prototype study w'asvunder- 
taken to collect improved information for evaluating 
compensatory programs. In the remainder of this 
present school year the following activities will be 
completed. A post-test will be administered to all 
grade five students in the ten community districts in 
New York City which are included in the Department's 
sample. In addition, an '^experimental version*^ of 
the reading effectiveness measure will be administered 
to all students. In addition, the results of these tests 
(including the pre- test scores) will be used along with 
detailed information collected on variables for each 
compensatory program to conduct the following analyses: 
1) a calculation of the actual growth of students in each 
program and an evaluation of the results relative to the 
growth expected on the basis of actuarial data obtained 
from a national norming sample; 2) a calculation of the 
extent to which students are able to comprehend text 
equivalent in difficulty to that which they are using in 
instruction; and 3) data will be reviewed to determine 
the feasibility of analyses of the following types: (a) pro- 
gram versus no program designs; (b) within-district 
program comparisons; (c) using both Basic Education 
Data (BEDS) - derived indicators and data from the 
Categorical Funding Study described previously for 
making between-district comparisons. The availability 
of this wealth of information for tho first time will 
make the selection of specific studies to be undertaken 
a difficult one. 



'2) Evaluation of U rban Education Projects - School Year 
197^ "73: Under question ^1, this summary has already 
provided most of the^ information regarding the rationale 
for the procedures uiaed and the results of Urban Educa- 
tion projects. To answer question ^2- -What types of 
projects are most effective for students of different 
characteristics? - -pages lOO-lll of Chapter II provide 
a summary of ^'exemplary^' projects. 

3) Urban Education Performance Indicators Study - School 
Year 1972-73 ; For several years, the Department has 
been developing the Performance Indicators in Educa- 
tion (P. L E» ) project as a statistical method of determini 
how various characteristics of the district relate to 
achievement. Generally, the data used in the statistical 
model has been BEDS (Basic Education Data System) 
information and PEP {Pupil Evaluation Program) scores. 
As part of the evaluation for the present year, this worK 
is being extended to examine specific Urban Education 
projects. The purpose is to develop Urban Education 
performance indicators to determine how various 
characteristics of projects relate to achievement. 

4) Study of the Characteristics of High Performing Upstate 
Schools ; The purpose of this component which will be 
developing in the coming year, is to identify character- 
istics of upstate schools which are highly successful 

in promoting student achievement given the conditions 
under which they operate and to identify practices in 
schools which are correlated with higher than anticipated 
student achievement in the basic skills. 

5) Study of th e Effect of Pupil Mobility Upo n Pupil Evalua- 
tion Program (PEP) R esults; Each year, after the PEP 
test results have been reported to the schools, it is not 
uncommon to have principals complain that because they 
have a large pupil turnover, they arc being unfairly 
'^blamed" for the educational deficiencies of pupils who 
have been in their schools for only relatively short 
periods of time. 

Of course, what the general public fails to realize 
is that it is not at all a matter of ''blame'\ The primary 
purpose of the PEP progran^ is lo identify and locate 
a target group of pupils for special educational prograins 
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If pupils are properly classified in the target group, 
on the basis that their scores fall below the Statewide 
Reference Point established for that purpose, how long 
the pupils have been enrolled in the school is not a 
particularly important consideration for purposes of 
educational planning. The point remains that students 
in the target grojup have certain educational needs, and 
the number of target group pupils in a school is an index 
of unmet needs. The validity of this index iii in no way 
diminished by the fact that more or less of the pupils 
are transferees. | 

Nevertheless^ it is true that there has been a 
tendency in some sectors of the press and the public 
to equate the size of the target group in a particular 
school with the effectiveness of the educational program. 
Certainly that error would be compounded if pupil 
mobility were ignored. It was determined then that it 
would be useful to explore the dimensions of the mobility 
problem, and the relationship between pupil mobility 
and PEP scores. 

The results of this study indicate the following. 
The frequency of transfer students, defined as students 
who began enrollment in Grade 1 in another school, is 
fairly high even by the time that the students reach 
Grade 3. In the State as ii whole, it is estimated that 
about 29% of the Grade 3 students tested in the October 1972 
PEP program were transfer students. As a group, the 
transfer students did in fact tend to have lower PEP 
reading test scores than the non-transfer students. How- 
ever, this student mobility factor did not seem to have 
a sufficiently differential effect on the PEP test results 
of schools in the same community type to affect the 
relative standing of individual schools. That is, if 
transfer students had been eliminated from the PEP 
reports, the relative standings of individual schools 
in Grade 3 reading scores^ as compared with other 
schools of the same community type, would have re- 
n^ained virtually unchanged. 
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Description of Progress in Developing the Projected Program; 
A Comprehensive Student Evaluation Program (Chapter III) 

a. Review of tvvo ''State of the Art" Problems--(l) Existing 
Testing Methodologies; Is the Educational System Succeed- 
ing, and, (2) Availability of Disaggregated Information 

The Department, after analyzing existing testing method-^ 
ologies, has concluded that standardized tests are not as 
suitable as new forms of criterion-referenced tests for 
determining whether New York Staters elementary and 
secondary students are successfully mastering basic skills. 
In addition, the Department is convinced that computer 
support systems are needed to be able to determine 
which instructional programs are more successful than 
others and to assist Local school districts in improv- 
ing instructional program planning, management and 
evaluation practices, 

b. Question #1: To what extent are New York State's elementary 
and secondary students successfully mastering basic skills? 

1) Desired Purposes of Tests ; The Department has con- 
cluded that three levels of tests are required to fulfill 
educational decision-making needs* First, there is 

a need to determine how well a student performs on 
tasks that may be encountered in adult life. Second, 
the teat(s) must be able to measure progress toward 
adult competence over grades, and also be sensitive 
enough to measure individual and program progress 
within a period such as a school year, Third, there 
is a need for test(s) to verify that specific subskills 
result in a given level of competence at a specified 
point in time, and to verify the most effective sequences 
in which subskills can be acquired. 

2) Progress in Developing More Suitable Measurement 
Devices ; The emphasis of the work completed or in 
progress regarding the first two levels of testing needs 
described above has been in the priority area of read- 
ing. A one-year research effort has been completed. 
The results of this effort were: (1) a determination of 
the properties desired in the new testing methodology; 
(2) a description of the development tasks which must 
be undertaken to develop a new reading effectiveness 
measure; (3) a review of existing approaches to the 
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measurement of the effectiveness in reading, both in 
terms of the properties desired and the technical prob- 
lems associated witli meeting the specified properties; 
and (4) a proposed work plan for developing the actual 
tests. An experimental version of a new reading effec- 
tiveness measur* will be administered to all grade five 
students in ten New York City community districts in 
the spring of 1974. To continue the development work 
on the reading effectiveness measure, the Department 
has submitted a two-year proposal to the National In- 
stitute of Education (N. I, E. ). Other funding will be 
sought this spring if N, I, E. funding is not secured. 
If additional funds are available, work on a similar 
measure for mathematics will be undertaken. The 
Department believes that the development of these tests 
will be useful in identifying some of the factors contri- 
buting to less -than-desired levels of effectiveness. 
A review of analyses possibilities is included in 
Chapter III. 

The Department^is making significant progress 
in implementing criterion-referenced tests to satisfy 
the need for tests which measure subskills, the third 
level of testing identified. This is being accomplished 
by implementing the Instructional Evaluation System 
described in a subsequent section. 

3) Plan to Satisfy the Immediate Need for Bas^c Competency 
Examinations ; The reading effectiveness measure will 
not be operational for two years because of the necessary 
work required to sample adult and school materials, 
develop an improved readability criterion scale, and 
an improved predictive readability formula. Conse- 
quently, the; Department has examined short term 
programs because it has committed itself in the near 
term to producing basic competency examinations in 
reading and mathematics. 

One alternative would be to utilize a measure of 
minimum adult competence developed by the Educational 
Testing Service* A second alternative would be to 
utilize an experimental version of the reading effective- 
ness measure. With this alternative a sample of 
materials used in the school districts would be made 
and their readability calculated utilizing existing 
readability scales. The materials' readability would 



be accepted as a do facto standard. A third alternative 
being considered is utilizing instructional objectives and 
matching criterion- refe renced test items, A decision 
as to which alternative is to be used will be made shortly* 

4 ) Pilot Study of Instructional Criterion-Referenced Objective 
Tes ts; Throughout this report, the Department indicates 
that standardized tests are not as useful as criterion- 
referenced tests at the instructional level. To determine 
the feasibility of supporting criterion-referenced testing 
on a broad scale in all the schools in New York State, 
a pilot study has been undertaken during this school 
year (1973-74) in ten school districts with different read- 
ing program needs in grades 4-6. 

The study has already yielded some important data 
on the feasibility of using criterion-referenced testing 
on a statewide basis. It is now clear that the partici- 
pating districts have largely selected the same instruc- 
tional objectives in the areas of vocabulary and compre* 
hension. Where the districts generally vary is in the 
"content'* associated with the objectives and in the 
number of testing levels (e*g. levels 4-6) applicable to 
the children in a given classroom or building, The 
reading objective bank developed by the Department 
appears to be adequate for the task and many useful 
test items are being developed which can help other 
districts avoid doing this work into the future. In 
1974-75^ this study will be expanded in the same districts 
to cover grades 1-6 in reading. 

Question #2: What types of instructional programs (or 
projects in the case of categorical aid programs) are 
most effective for students of different characteristics? 

1) Instructional Evaluation System ; The Department has 
made meaningful progress in developing and imple- 
menting the Instructional revaluation System, This 
System consists of two major components which when 
implemented together in any local school district 
make it possible to determine what instructional 
programs are more successful than others. 
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The first component of the system is a set of 
procedures at the local level for: (1) systematically 
developing instructional program objectives that can 
be operationalized across classrooms; (2) developing 
or assembling instructional activities and criterion- 
referenced test items linked to instructional program 
objectives; (3) measuring performance with test items, 
and interpreting test scores using predetermined per- 
formance criteria; (4) systematically reallocating 
instructional resources (e.g. materials, teacher time) 
to improve student performance; and (5) systematically 
recording instructional activities, objectives, and test 
scores to provide a basis for upgrading instructional 
program content and evaluating instructional perform- 
ance over time. 

The second component is a computerized information- 
handling component which scores tests, keeps records 
on pupil and programs and generates reports for stu- 
dents, parents, teachers and educational decision- 
makers. Two computerized software systems are 
presently being used to implement the Instructional 
Evaluation System--the Instructional Support System 
(I.S.S. ) which was developed and implemented in one 
New York State upstate district and the Comprehensive 
Achievement Monitoring (C. A. M. ) system which was 
developed at the national level. 

The Department has decided to use the I.S.S. in 
New York City and C. A.M. in upstate and Long Island 
districts. The Instructional Evaluation Sys tem must 
be implemented gradually because of the work involved 
for any given district. In the present year the highest 
priority is to begin working with New York City commu- 
nity districts. One district has already committed it- 
self to implementing I.S.S, in all of its thirteen ele- 
mentary schools for mathematics in grades 1-6. Other 
community districts are also considering utilizing the 
system. In upstate and Long Island districts, C. A. M. 
is already being used in many districts. Efforts are 
underway to expand the coverage through B.O.C.E.S, 
personnel. 

This report includes some proliminary data indi- 
cating that the systeu) is having an irT\pact. (See 
pages 162-166). 
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Question ffi: To what extent are the best instructional 
planning, management and evaluation practices being 
used at different levels of the educational system? 

1) Instructional Evaluation System ; The implementation 
of the Instructional Evaluation System has provided 
the Department with data concerning how instructional 
planning, management and evaluation practices can 

be improved. It is clear that the availability and use 
of the two components described above leads to an 
examination of how existing practices at the local 
school district level should and can be improved. 
Findings in this regard will be made available in 
subsequent reports. 

2) Categorical Funding Study ; The Department has pre- 
liminary findings from -a special study which is designed 
to systematically examine instructional planning, man- 
agement and evaluation practices in a sample of school 
districts presently receiving funds for compensatory 
education programs (i.e. E.S.E.A. - Title I and Urban 
Education). The Department believes that the recom- 
mendations in a final report in the fall of 1974 will be 
generally applicable to all educational practices, not 
just to those connected with compensatory efforts. 

The goal of the study is to recommend feasible, 
cost-effective means for the State Education Depart- 
ment to improve the planning, management and evalua- 
tion of compensatory programs for the educationally 
disadvantaged. There are six components to the study 
which are reviewed in Chapter II. Preliminary findings 
of one component are presented in some detail. These 
findings have to do with the project proposals and eval- 
uation reports submitted to the Department for approval. 
These findings will be analyzed further after field inter- 
views are completed. The scaling of the project appli- 
cations with measures (called application and evaluation 
quality measures) attempted to determine how district's 
planning, management and evaluation practices (as sub- 
mitted on proposals and evaluation reports) scored in 
relationship to contemfforary planning, management and 
evaluation standards. 



The scores indicate that there has been an upward 
trend in application quality over the four -year period 
studied for all district types and for both the Urban 
Education and E.S.E.A. Title I projects. Similar 
improvement was found in evaluation scores for 
New York City community districts. Evaluation scores 
for large and moderate upstate districts remained 
relatively stable over the period of the study. Despite 
the existence of these improvements in scores, how- 
ever, the scores were still found to be quite low when 
compared to contemporary planning, and evaluation 
standards. This indicates that the final recommenda- 
tions of the study will have to deal with this finding 
if through field interviews these initial findings are 
verified. 



11. DESCRIPTION OF PROGRESS TOWARD 
THE INTERIM PROGRAM 

In the introduction to this i^eport, the summary of the two '^state of the 

art" problem areas --existing testing methodologies and the availability of 

disaggregated information-vindicated that the Department is constrained in its 

ability in the short run to answer completely the three performance -related 

questions being asked of the educational system by the Governor, the Legis- 
lature, the public at large and educators themselves for current categorical 

and general educational programs. The Department's efforts described in this 

Chapter, however, even though not as sui*^able as those described in Chapter III 

the projected program, have dramatically improved over those reported in the 

past. The Department is confident that it has implemented an interim program 
which makes the best possible use of existing methods and the most s ys tematlc 
ujie uf tjxtstliig liifoimatLori to anoWci* the three qucGtlons addres?eH in the intro 
duction. 

In this chapter work efforts will be reviewed under the two performance- 
related questions used in the previous chapter. The third question- -'*To what 
extent are the best instructional planning, management and evaluation practices 
being used at different levels of the educational system? ^'--is discussed in 
Chapter IIL 

Under question #l"-*^To wha t extent are New York Staters elementary 
and secondary students successfully mastering basic skills? -the follow- 
ing work efforts are reviewed in this section: 1) Study of Local School 
Districts^ Nationally Standardized Test Programs; Z) Comparison of 
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Regents Examination Results in Biology, English, French, Geometry and 
Social Studies with Resulti from Commercially -Published Standardized 
Tests; 3) New York City Prototype Study of Urban Education and Elementary 
and Secondary Education (E.S.E.A. ) - Title I Programs - School Year 1973-74; 
4) Evaluation of Urban Education Projects - School Year 1972-73; and 5) Test- 
ing to Measure the Effectiveness of Bilingual Education. 

Under question #2-»>^^What types of instructional programs (or projects 
in the case of categorical aid programs) are most effective for students of 
different characteristic s? " - -the following work efforts are reviewed: 
1) New York City Prototype Study of Urban Education and E.S.E* A, - Title I 
Programs - School Year 1973-74; 2) Evaluation of Urban Education Projects 
- School Year 1972-73 (Exemplary Projects); 3) Urban Education Performance 
Indicators Study - School Year 1972-73; 4) Characteristics of High Performing 
Upstate Schools; and 5) Study of the Effect ot Pupil Mobility Upon Papil 
Evaluation Program (PEP) Results. 

The following sections review each of these efforts under each of the two 
questions, 

A. QUESTION #1; TO WHAT EXTENT ARE NEW YORK S TATE'S 
ELEMENTARY AND SECONDARY STUDENTS SUCCESSFULLY 
MASTERING BASIC SKILLS? ' ^ 

In reviewing the efforts described below it is important to keep in mind 

the reasons why each of these work efforts has been included in this chapter 

describing the interim program. There are, as indicated in the introduction, 

barriers imposed by testing methodologies and the unavailability of disaggregated 
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Question ff It To what ent .21- 
are New York State's elementary 
and secondary students successfully 
mastering basic skills? 

information. These barriers will not be reviewed again here but they should 
be considered in reading what follows. It should also be noted that some of 
the statistical procedures used here and under question #2 are presently being 
reviewed in terms of their appropriateness. 

!• Study of Local School Districts* Nationally 
Standardized Teat Programs 

a. Purposes of the Study 

New York State, with its Pupil Evaluation Program 
(PEP), is one of the few states that has mandated adminis- 
tration of uniform State-developed testo. The PEP tests 
are administered annually in reading and mathematics in 
all schools at the beginning of grades 3, 6 and 9. Results 
from these tests are widely reported. In addition, many 
Local school districts administer nationally standardized 
tests. These tests are used at the local level for various 
purposes including that of determining how well their 
students are mastering basic skills. 

For year* many individuals both within and. outside 
the Department have expressed a desire to know what 
specific tests are being used in local districts, at what 
time during the school year they are administered, how 
many districts administer such tests, what the cost is 
to the schools, etc. The Department^ therefore, designed 
and implemented a study to address these questions. The 
specific purpose was to obtain information on the scope 
of the evaluation activities now going on in the local 
school districts, and the extent of the total financial 
commitment to such evaluation programs. While there 
has been inc reased public demand in recent years for 
more evaluation and accountability at the State level, it 
was hypothesized in undertaking this study that there has 
been inadequate recognition of the extent to which local 
school districts have been actively and continuously 
concerned with evaluation. 
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Quesdon #1 ; To what extent are New York Staters elementary and secondary 
students succeasfuUy mastering basic skills? 

A second purpose of tlie study was to explore the 
potential of using local school results on nationally 
standardized tests as an alternative to the PEP testing 
program. The major purposes of PEP are to help identify 
and locate the low-achieving students in the State and to 
determine trends in the general level of achievement from 
year to year. While the PEP tests have heretofore not 
permitted comparison of, the level of achievement in 
New York State with national norms, comparability studies 
for that purpose are now under way, so that a new and 
additional subpurpose of the PEP tests may be considered 
to be a comparison of New York State test scores witli 
national norms. The key question, of course, is whether 
local school tests would be equally as effective as the PEP 
results in serving the particular purposes ior which the 
PEP tests were designed. 

It may be that, despite variations in^tests used by 
local schools, the grade level scores of the students so 
obtained will identify low -achieving students as effectively 
as the PEP. Also, despite variations in practice that 
may exist with respect to the particular tests, or the 
time of year at which the tests were administered, it is 
possible that the results may be sufficiently comparable 
so that if they are integrated on a systematic basis, 
they will provide a useful picture of the trend in Statewide 
achievement levels from year to year, or in comparison 
with national norms. 

If that were found to be the case, local tests would 
have certain advantages over PEP. Local tests could 
provide data for all grades rather than merely three 
grades, they would be more comprehensive in scope 
and probably better suited to evaluating school prograrni^, 
they would permit greater flexibility with respect to date 
of testing, and duplicate testing would be reduced. 

Accordingly, the survey of local school testing included 
an analysis of the test results as well as the testing practices. 
To accomplish this second purpo.sc of the survey, however, 
further analysis will be nccessciry in order to '^termine 
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Question #1; To w hat e xtent are New York State's elennentary and secondary 
students succes sfully mas tering basic skills? 



comparability with results provided by PEP. In par- 
ticular, when the PEP comparability study {see page 
35 ) is completed in May, it will be possible to 
verify whethe r a Statewide g rade-ievel obtained by 
pooling a variety of different local school tests admin- 
istered at different times to varying percentages of 
students would be comparable to the Statewide grade- 
level obtained from the PEP tests, 

b. Procedures Used in the Study 

During the first week of September 1973 official report forms 
were sent to all public and nonpublic schools in the State re- 
questing that they report the names of standardized tests of 
reading and mathematics adnainiatered gradewide during the 
school year 1972-73 in any of grades 3 to 8, inclusive, along 
with the numbers of pupils tested, the dates administered, 
and the school medians for total scores. At the same time, 
public school districts were requested to respond to a brief 
questionnaire covering standardized achievement test ex- 
penditures and reporting practices for grades K-12^in the 
school district as a whole* 

The requested return date for the completed forms was 
October 12, 1973, Schools and school districts were directed 
to submit reports even if no tests had been administSred. 
Where returns were not received by the due date, follow-up 
was made by telephone or by letter. However, a cut-off 
date of November 30 had to be established for purposes of 
computer processing. 



c. Results from the Portion of the Studv Completed to Date 

1) Extent to which Schools Administer Nationally Standa id - 
ized Tests: Tables 1 and 2 show that 3, 177 public schools, 
90% of those returning report forms, administered grade 
wide one or more standardized tests of reading to pupils 
in their grade 3-8 enrollments, and 3,037 (867o) admin- 
istered one or more in mathematics. These schools 
tested roughly 75% of their combined grade 3-8 enroll- 
ment with a reading test and 61% with a mathematics 
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TABLE 1: Amount of Standardized Reading Testing during 1972-7 3 
in Grades 3-8 by Publi c Schools Returning Report For ma 





Schools Returning Forms 


Schools Reporting Results 






% Of 


Gr. 3-8 


% of 






No. 




Grades 


No. 


Total 


Enroll . 


Total 


No. 


% , 


Tested 


% 


3 


2700 


95% 


259,789 


96% 


2050 


76% 


193,19.1^ 


74% 


4 


2637 


95 


261,810 


96 


2300 


87 


218,596' ' 


83 


5 


2563 


95 


259,757 


96 


2180 


85 


211,989 


82 


6 


2226 


95 


257,926 


96 


1867 


84 


205, 220 


80 


7 


967 


91 


250,777 


94 


636 


66 


161,429 


64 


8 


946 


91 


240,995 


94 


560 


59 


. 150,803 


63 


Comb. 


3535 


94% 1, 


531,054 


95% 


3177 


90% 


1,141,229 


75% 



TABLE 2: 



Amount of Standardized Mathematics Testing during 1972-73 



in Grades 3-8 by Public Schools Returning Report Forms 



Schools Returning Forms 



Schools Reporting Results 







% of 


Gr. 3-8 


% of 






No. 




Grades 


No. 


Total 


Enroll. 


Total 


No. 


Ji- 


Tested 


% 


3 


2700 


95% 


259,789 


96% 


1916 


71% 


178,152 


69% 


4 


2637 


95 


261,810 


96 


1599 


61 


132,918 


51 


5 


2563 


95 


259,757 


96 


2074 


81 


200,521 


77 


6 


2226 


95 


257,926 


96 


1817 * 


82 


200, 264 


78 


7 


967 


91 


250,777 


94 


38] 


39 


80,961 


32 


8 


946 


91 


240,995 


94 


500 


53 


132, 313 


57 


Comb . 


3535 


94% 


1,531,504 


95% 


3037 


86% 


930,129 


61% 
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Question if It To what ext e nt are New York State's elementary and secondary 
students successfully mastering basic skills? 

Of the school s administering tests^ the hij^hest per- 
centage administer reading tests in grades 4, 5 and 6 
(87%^ 85%; 84%); and mathematics te st S; in grades 5 
and 6 (81% and 82%). Fewer schools administer tests 
in grades 7 and 8t In these grades^ reading tests are 
admini stered by 66 % and 59%, and mathematics tests 
by only 39% and 5 3% > The percentages of pupils tested 
in the individual grades in these schools are generally 
lower but have a pattern roughly comparable to the per- 
centages of schools using tests. 

Tables 3 and 4 provide an analysis of the amount of 
testing in public schools by type of community. The combined 
community type totals in Table 3 show that schools in most 
every type, except Large City (77%), administer a standard- 
ized test in reading to pupils in one or more grades of their 
grade 3-8 enrollments. The New York City schools admin- 
ister a reading test in each grade in practically every school, 
but in schools in other community types practices vary. 
For example, schools in medium size cities administer 
reading tests in most every grade except grade 7; the per- 
centages for these medium size city schools range from 
85% to 99% in grades 3-6 and grade 8, but only 52% admin- 
ister a reading test in grade 7. In large cities, 81% of the 
schools administer a reading test in grade 7, but only 33% 
in grade 8. In village -suburban and rural areas, schools 
tend to test reading in each of grades 3-6, but only half 
of the schools in these types test reading in grades 7 and 
8 (Vill-Sub. , 59% and 56%; Rural, 50% and 46%). 

In mathematics (Table 4) as in reading, the combined 
totals for community type show that most schools, except 
those in large cities, adminif^ter a rnatheinn lies tef^l i^i 
one or more of the grades in tlicir grade 3-3 eu * oUments. 
The grade-by-grade patterns \% ithin c on :i i lui ty types are 
similar to those for reading, but generally show slightly 
less gradewide testi ig in nathemalics than in reading. 
For example. New Y ?rk City schools administer mathematics 
tests in grades 3, 5, 6 and 9, only; no gradewide tests are 
administered in grades 4 and 7. 
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TABLE 3; Number of Schools Administering Nationally Standardized 
Tests of Re adin g during 197 2-7 3 and the Percentage Based on Number 
of Schools R etur ning Report Forms by Grade and Community Type 



Grade NYC Lq.City 

T hird Grade 

No. schools 615 103 

% of Tot. 99% 45% 

Fourth Grade 

No. Schools 613 158 

% of Tot. 99% 70% 

Fifth Grade 

No. schools 609 136 

% of Tot. 99% 62% 



Med. City Sm.City Vill>.Sub Rural Combined 



81 
85% 



95 
99% 



94 

98% 



160 
64% 



206 
86% 



190 

83% 



705 



773 
80% 



386 



835 . 393 
83% 88% 



378 
87% 



2,050 
76% 



2,300 
87% 



2,180 
85% 



sixth Grade 
No. Schools 502 
% of Tot. 98% 



102 
50% 



92 



161 
87% 



644 

80% 



366 
87% 



1,867 
84% 



Seventh Grade 
~No. Schools 171 70 
% of Tot. 98% 81% 



13 
52% 



40 
71% 



185 
59% 



157 
50% 



636 
66% 



Eighth Grade 
No. Schools 158 28 
% of Tot. 98% 33% 

Combined 

No. Schools 783 210 
% of Tot. 99% 77% 



24 

96% 



110 
99% 



34 
61% 



274 

89% 



174 

56% 



1261 
89% 



142 
46% 



549 

85% 



560 
59% 



3,177 
90% 
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TABLS 4 ; Number of Schools Administering Nationally Standardized 
Tests of Mathematics during 197 2-.73 and the Percentage Baaed on Number 
of- Schoo ls Returning Report Forms by Grade and Community Type 



Grade NYC Lq.City Med. City 

Third Grade 

617 99 
99% 43% 



Sm.City Vill-Sub Rural Combined 



No. Schools 
% of Tot. 



Fourth Grade 

No -Schools 0 

% ot Tot. 0% 

Fifth Grade 

No. Schools 610 

% of Tot. 99% 

Si xth Grade 

No. Schools 503 

% of Tot. 98% 



Seventh Grade 
No. Schools 
% of Tot. 

Eighth Grade 
No . Schools 
% of Tot. 

Combined 
No . Schools 
% of Tot. 



0 



160 
99% 



784 



154 
68% 



136 
62% 



103 

50% 



62 
72% 



23 
27% 



209 

77% 



66 
69% 



90 
94% 



79 
82% 



87 
91% 



10 
40% 



19 

76% 



103 
93% 



144 

57% 



194 
81% 



175 
76% 



153 
83% 



30 
54% 



29 
5 2% 



256 
83% 



631 
60% 



616 



152 
49% 



152 
49% 



1168 
83% 



359 
79% 



780 381 
77% 85% 



704 370 
73% 85% 



355 
84% 



127 
41% 



117 
38% 



517 

80% 



1,916 
71% 



1,599 
61% 



2,074 

81% 



1,817 

82% 



381 
39% 



500 
53% 



3,037 
86% 
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Question #1; To what extent are New York Staters elementary and secondary 
s t\ I de nts auccessfuUy mastering basic skills? 

2) Time of Year in which Schools Administer Nationally 
Standardized Tests ; Tables 3 and 6 provide informa- 
tion on the months during which public schools admin- 
ister nationally standardized tests of reading and mathe- 
matics. Most of the 3, 177 schools reporting reading 
reading test results for one or more of grades 3-8 ad- 
minister nationally standardized tests of reading in 
the springy 34% in April, 37% in May and 7% in June 
--a total of 78%. About 28% ot the schools administer 
tests in the fall, 9% in September, 16% in October and 
3% in November , The combined percentages, of course, 
add to more than 100% because some schools administer 
two or more reading tests per year on a gradewide basis 
and a school is included in each of the months in which 
it administered a test. These 3, 177 schools were 90% 
of the 3, 535 returning report forms and 85% of the 
3,751 schools with enrolhnents in these grades. 

Of the 3, 037 schools reporting mathematics test 
results, most schools generally administer mathematics 
tests at times during the year similar to those for 
reading, but with one notable exception. Although , 
as for reading, a high percentage of schools administer 
mathematics tests in the spring, only 9% administer 
them in April and 60% in May as compared with 34% 
and 37% administering reading tests in these months . 
These 3, 037 schools were 86% of the 3, 535 schools 
returning report forms and 81% of the 3,751 schools 
with enrollments in these grades. 
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TABLE 5; Percent of 3,177 Public Schools Reporting Reading Test 
Results by Month of Year in Which They Administered the Tests 



Month 


Gr.3 


Gr.4 


Gr.5 


Gr.6 


Gr.7 


Gr.8 


Combined 


(No 


( 2050) 


( 2300) 


{ 2180) 


(1867) 


(636) 


(560) 


( 31 77) 


September 


8% 


8% 


7% 


7% 


7% 


6% 


9% 


October 


9 


13 


13 


12 


19 


18 


16 


November 


2 


3 


2 


3 


3 


4 


3 


December 


0 


0 


1 


0 


2 


2 


1 


January 


2 


2 


2 


4 


4 


5 


4 


February 


2 


2 


2 


4 


2 


2 


3 


March 


3 


4 


4 


3 


2 


2 


5 


April 


36 


35 


36 


36 


32 


34 


34 


May 


39 


36 


34 


33 


27 


26 


37 


June 


7 


5 


6 


5 


7 


6 


_2 


Total 


10®4 


•108% 


107% 


107% 


105% 


105% 


119% 



1 Possible for total percentages to exceed 100% because schools 
administering tests more than once a year, e.g., fall and 
spring, were counted each time tests were administered* 



TABLE 6: Percent of 3.037 Public Schools Reporting Mathematics Test 
Results by Month of Year in W-hich They Administered the Tests 



Month 


Gr. 3 


Gr.4 


Gr.5 


Gr.6 


Gr.7 


Gr.8 


Combined 


(No.=) 


(1916) 


(1599) 


(2074) 


(1817) 


(381) 


(500) 


(3037) 


September 


7% 


10% 


7% 


6% 


5% 


4% 


8% 


October 


10 


18 


14 


12 


24 


18 


16 


November 


1 


4 


2 


3 


5 


4 


3 


December 


0 


1 


1 


0 


3 


2 


1 


January 


2 


3 


2 


4 


5 


5 


4 


February 


1 


2 


1 


3 


1 


2 


3 


March 


3 


5 


3 


4 


• 3 


2 


5 


April 


7 


11 


8 


9 


7 


5 


9 


May 


69 


47 


61 


58 


41 


55 


60 


June 


6 


7 


5 


5 


8 


4 


5 


Total (1) 


106% 


108% 


104% 


104% 


102% 


101% 


114% 



Possible for total percentages to. exceed 100% because schools 
administering tests more than once* a year, e.g., fall and 
spring, were counted each time tests were administered. 
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Qoes^lion I; To wh^t extent are New York State 's elcmentaty and s e c onda r 
students succ essfully maste ri ng basic skills? " " 



Nationally Standardiz e d Tests Common ly Used by 
Schools: Tables 7 and 8, giving the percentages of 
schools using various naUonally standardized tests 
of reading and mathematics, show that the most 
widely used tests in the State are the Metrop _olUan 
Achievement Tests (40% and 41%), the Iowa Tests 
of Basic Skills (31% and 33%) and the Stanfor d 
Achievement T ests (19% and 19% ). This is true for 
each of grades 3-8 and for all grades combined. 
All other tests combined have less than a 20% usage 
rate. 



TABLE .7;, Percent of 3,177 Public Schools Reporting Reading Tf^sh 

Results by Name of Standardized Test Admin istered 



Name of 



Test (1) 


Gr. 3 


Gr. 4 


Gr. 5 


Gr. 6 


Gr. 7 


Gr. 8 


(No. =) 


(2050) 


(2300) 


(2180) 


(1867) 


(636) 


(560) 


MAT 


44% 


41% 


40% 


39% 


41% 


39% 


ITBS 


23 


31 


29 


31 


24 


29 


SAT 


19 


17 


18 


18 


14 


12 


CAT 


5 


5 


5 


6 


6 


5 


Gates-M 


5 


4 


4 


4 


6 


6 


CTBS 


3 


3 


3 


2 


1 


1 


SRA 


2 


1 


2 


2 


3 


3 


STEP 


0 


0 


0 


0 


1 


2 


Other 


2 


1 


2 


2 


6 


5 


Total (2) 


103% 


103% 


103% 


104% 


102% 


102% 



Combined 
(3177) 

40% 

31 

19 

5 

6 

2 

2 

1 

3 

109% 



MAT/Metropolitan Achievement Tests; ITBS/Iowa Test of Basic 
Skills; SAT/Stanford Achievement Test? CAT/Calif ornia Achievement 
Test; Gates-M/Gates-MacGinitie; CTBS/Comprehensive Test of 
Basic Skills; SRA/SRA Achievement Series; STEP/Sequential Tests 
of Education Progress 



2 Possible for total percentage to exceed 100% because schools 
administering two or more different tests in the same grade 
during the year, e.g. MAT in the fall and ITBS in the spring, 
were counted each time a different test was administered. 
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TABLE 8; Percent ages of 3.037 Public Schools Reporting Mathematics 
T est Results bv Name of Standardized Test Administered 

Name of 



Test 


Gr. 3 


Gr. 4 


Gr. 5 


Gr. 6 


Gr. 7 


Gr. 8 


Combined 


(No. =) 


(1916) 


(1599) 


(2074) 


(1817) 


(381) 




\ OV ^ f f 


MAT 


46% 


20% 


41% 


39% 


20% 


42% 


41% 


ITBS 


25 


44 


31 


32 


42 


32 


33 


SAT 


20 


23 


18 


19 


22 


15 


19 


CAT 


5 


7 


5 


5 


9 


5 


5 


CTBS 


3 


4 


3 


3 


2 


1 


2 


SRA 


2 


2 


2 


2 


3 


3 


2 


STEP 


0 


1 


0 


0 


1 


2 


1 


Other 


1 


2 


1 


1 


1 


1 


1 


Total < 2) 


102% 


103% 


101% 


101% 


100% 


101% 


104% 



1 MAT/Metropolitan Achievement Tests; ITBS/Iowa Test of Basic 
Skills; SAT/Stanford Achievement Test; CAT/California 
Achievement Test; CTBS/Comprehensive Test of Basic Skills; 
SRA/SRA Achievement Series;STEP/Sequential Tests of 
Educational Progress 



2 Possible for total percentage to exceed 10096 because schools 
administering two or more different tests in the same grade 
during the year, e.g. MAT in the fall and ITBS in the spring, 
were counted each time a different test was administered. 
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Questlo n Hi To what extent are New York State's elementary and secondary 
st u dents successfully mastering basic skills? 



■i) School District Expenditures for Nationally Sta ndard ized 
Achievement Tests ; Expenditures for standardized 
achievement test materials and services. during 1972-73 
for grades K-12 by public school districts in the State 



\yere a little over 2.4 million dollars (Table 9)_» This 
estimate is based on reports received from 95% of all 
the public school districts in the State and assumes 
districts from which no reports were received had 
expenditures similar to those reporting within the 
same community type. The average expenditure 
statewide amounted to $. 70 per pupil enrolled. Among 
community types, the per pupil enrolled expenditures 
ranged from lows of $.61 and $.62 in rural and large 
city districts to highs of $. 73 and $. 74 in .New York 
City and Village -Suburban districts. 



'^^^E ^- public Scho6l District Expenditures for Standardized 

Achievement Tes t Materials and Services in Grades K-12 d uring 1972-73 



Community 
Type 

New York City 
Large City 
Medium City 
aiiall City 
Vi 11- Suburban 
Rural 

Statewide 



No. of 

Pi striata 

31 
7 
8 

46 
263 
403 

758 



K-12 Total 

m ent Expenditures ^^-^ 

(2) 



1/122,788(2) 
204, 394 
85,099 
218,988 

1, 398,014 
443.935 



? 814,173 
126,843 
59,647 
140,695 
1,032,419 
270.707 



3,473, 218 $2,444,484 



Ave . Expend . 
Per Pupil 
Enrolled • 

$.73 
.62 
.70 
.64 
.74 
.61 

$.70 



(1) Assumes that school districts not reporting (5% statewide; 
Table 1) had per pupil expenditure rates comparable to those 
in other school districts in their community type. 

(2) Includes total Central Board enrollments and expenditures 
as well as those for the 31 local districts 
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Question HI: To what extent are New York Staters elementary and secondary 
students successfully mastering basic skills? 

5 ) School District Practices in Reporting Summarized 
Test Results » It appears to be fairly common practice 
among school districts in the State to provide their 
school boards^ their administrators and their teacher s 
with summarized information on the results of New York 
State standardized tests (PEP) > As shown in Table 10 
the percentages are 81%, 89% and 83% respectively. 
Although roughly 90% of the school districts report using 
nationally standardized tests> only 40%, 55% and 49% 
pro vide their boards and administrative and teaching 
staffs with summarized results on these tests. 

It should be noted that even though summaries are 
not provided to the teachers, they do receive results 
for each child in their classroom. These results do 
indicate that a greater effort is required on the part 
of the school districts who do not presently report on 
their results in making information available. However, 
if the school districts conclude that the results from 
such tests are not useful for decision-making, a decision 
should be made to invest the monies now being spent 
in a testing program which does give them a capability 
to make improved decisions. 

About half (49%) of the districts provide their 
residents with summarized reports on State tests, 
and only 21% provide their residents with summarized 
information on national tests . These figures again 
indicate that school districts should make a greater 
effort in reporting to community. The State results 
appear to be more useful from their point of view. 
If this is the case they should make greater efforts 
to interpret these results for the community. 

Table 11 shows that the school districts that do 
report test results to district residents seem to utilize 
a variety of methods to accomplish this purpose rather 
than rely on any one particular method, and some 
apparently use a combination of methods. About 50% 
report using each of the three primary methods available 
to them: District Newsletter, public meetings and 
formal district releases to the news media. 
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li^J^-J^l Public School District Practices in Reporting Summari?:ed 

Resu_lts on State and Nationally Standardized Ach ievement Tests Du rii 
1972..73 by 720 (95%) of 758 School Districts Stc.tfiwir?P " 



OUUUuclJu J. KSSUXuS 

Reported to 


N.Y. state 


Tests 


National 


Tests 


No. of 

Districts 


% 


No. of 

Districts 


% 


Board of Education 


586 


81% 


289 


40% 


School Admini strator s 


640 


89 


398 


55 


Teachers 


596 


83 


356 


49 


District Residents 


352 


49 


152 


21 



TABLE 11; Methods Used to Report Sv^mmarized Results on State and 
National Standardized Tests to School District Residents b y the 3 52 Districts 
Reporting Results on State Tests and the \SZ Districts Reporting Resvilts 
o n"fvfationaI Tests during 1972-73 (Table 10) 





N.Y. State 


Tests 


National 


Tests 


Reporting Methods 


No. of 

Di stricts 


% 

{N = 352) 


No. of 
Districts 


% 

(N=152) 


District Newsletter 


168 


47% 


66 


43% 


Public Meetings 


192 


54 


86 


57 


Fotmal District Release 
to News Media 183 


51 


76 


50 
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Question #1; To what extent are New York State's elementary and secondary 
students successfully mast e ring basic skills? 

d. Description of the PEP/Standardized Test Comparability Study 

The PEP/Standardized Test Comparability Study has two 
major purposes: 1) to compare the achievement of pupils 
in New York State with the achievement of pupils nationwide 
in reading and mathematics at the beginning of grades 3 
and 6; 2) to determine the correlations between the New York 
State PEP tests and commercially-published standardized 
tests in reading and mathematics for grades 3 and 6. 

As a by-product of the first of these purposes, compar- 
ability tables will be developed which will enable schools to 
interpret PEP test scores in terms of national norms, Here.- 
tofore, only New York State norms have been provided for 
the PEP tests. 



In order to accomplish these purposes^ a sample of 
schools has been asked to submit individual pupil scores 
in reading and mathematics obtained on the PEP tests for 
grades 3 and 6 and on one of the two most commonly used 
commercially-published standardized tests, the Iowa Tests 
of Basic Skills and the Metropolitan Achievement Tests, 
The tests are those administered in the fall of 1973 as a 
part of the regular school testing program; schools were 

not asked to do any additional testing for this study. When 
the scores have all been received, the sample will be as 
follows : 

\ No. of Pairs of Scores 

Comm e rcial Tests Grade 3 Grade 6 

Iowa Tests of Basic Skills 4,000 5, 500 

Metropolitan 3,000 2, 500 

These scores will be obtained from a representative sample 
of approximately 30 school districts. 

Comparability will be established by the equipercentile 
method. Scores having the same percentile rank in the 
equating sample will be considered comparable, permitting 
the interpretation of PEP test scores on the basis of the 
commercial test norms* Thus, it will be possible to compare 
the percent of pupils in New York State who fall below any 
given PEP test score with the percentage of pupils in the 
national norms sample who would fall below this score if 
^IjfeS;;^ the PEP test were administered nationwide. 



Question #1; To what extent are NTew York Staters elementary and seco ndary 
students successfully mastering basic skills? 

The correlation between the PEP tests and tlie commercial 
tests will indicate the extent to which the tests measure common 
factors. If the correlation is high and the tests are similar in 
content--. 90 or higher--the test results could be considered 
interchangeable^ and the same general interpretations about 
pupil achievement can be made from the PEP test results as 
from tlie results of the Iowa or the Metropolitan tests. 

e. Summary of the study 

Reports were submitted by 94% of all public scliools statewide 
enrolling 95% of all public school pupils state%vide* During 
the 1972-73 school year^ 90% of these schools administered 
a reading test in one or more grades, testing 75% of their 
total grade 3-8 enrollment; 86% administered a matj^emalics 
test in one or more grades, testing 61%of their enrollment. 
About 75% of the schools using standarciized tests report 
administering Lhem in the spring months as compared with 
25% in the fall. Most of the schools use the Metropolitan 
Achievement Tests, the Iowa Tests of Basic Skills, or the 
Stanford Achievement Tests. Eveii though most school 
districts did return questionnaires ret^arding the scope 
of their testing program, there are variations in: a) 
how much testing different types of districts athni nis te red; 
b) time of testings, and c) tests used. ^ 

School districts reported spending a total of roughly 
$2. 4 million on standardized achievement testing in 
grades K-ll during 1972-73, an average of about $* 70 
per pupil enrolled. Most school districts report summar)es 
of their State standardized test results to school board and 
school staff members and about half the districts report State 
standardized test results to district residents. Less than 
half the districts report nationally summarized test results 
to their school board and school :>taff members and only 
21% report nationally standardized test results to district 
residents. School districts that report test resultr. to 
district residents use district newsletters, public meetings 
and formal releases to news media with about equal frequency. 

Decisions regarding whether or not to substitute commer- 
cial standardized tests for the PE Program will be made 
after a substudy is completed in the spring which will determine 
the correlation between PEP and two national standardized 
tests e^j^d the similarity of content between the tests. 
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Comparison of Rege n ts Exaprnnabion Results in Biology , 
English, Frenc h , Geometry and Social Studies with Results 
from Commercially -Published Standardized Tests 

a. Purposes of the Study 

The administration of Regents examinations has a long 
history in New York State. Many educators and students 
have questioned whether they should be continued and 
others have suggested that commercially-published 
standardized tests replace Regents exams since they 
would allow the State to compare its students with national 
norms. The Board of Regents have concluded that the 
examinations should be continued until better examina- 
tions become available. The Regents believe that stand- 
ards inust be set at the State level even though Regents 
examinations are generally only taken by students with 
average and above-average scholastic ability. This study 
examines the feasibility of using the commercially-published 
tests in place of Regents examinations and also attempts to 
answer the following two additional questions: 

1) How do the achievement test scores of New York State 
Regents -level students compare with scores obtained 
by high school students nationwide? 

2) i-Iow closely correlated are Regents examination and 
standardized test scores in any given subject? 

b. Procedures Used in the Study 

The procedures used to answer the first question involved 
the following steps: 

1) Choosing five Regents examination subjects for study, 
and selecting commercially -published standardized 
tests in each of thes^e five subjects. 

Z) Administering each standardized test to a sample of 
students (reference group) preparing to take the corre- 
sponding Regents examination in June 1973. 

3) Establishing the comparability of selected scores in 
each pair of tests, Regents and standardized. 
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4) Finding the percentile ranks of these selected scores 
in two norm tables for each subject: (a) the norms 
for the June 1972 Regents examination^ and (b) Uie 
norms for tlie standardi^sed test. 

The five Regents examination subjects selected for this 
study were biology, English, French, tenth year mathematics 
(geometry), and social studies. The standardized tests selected 
we re J 

1) Nelson Biology Test, Revised Edition, Form E, 
published by Harcourt Brace Jovanovich, 1965. 

2) Stanford Achievement Test, High School Battery, 
Test 1; English, Form X, published by 
Harcourt Brace Jovanovich, 1965. 

3) Pimsleur French Proficiency Tests, Test 3: 
Reading Comprehension, Second Level, Form C, 
published by Harcourt Brace Jovanovich, 1967. 

4) Modern Geometry Test, published by Houghton 
Mifflin, 1971. 

5) Stanford Achievement Test, High School Battery, 
Test 6: Social Studies, Form X, published by 
Harcourt Brace Jovanovich, 1965. 

An effort was made to select tests which were reasonably 
similar in content to the corresponding Regents examinations, 
were recently standardized, and could be administered in one 
class period* In the case of English, French, and social studies, 
it was impossible to satisfy the first two of these criteria, al- 
though the tests finally selected came closer than any of the 
other tests which were available. 

The standardized tests were administered by the regular 
classroom teachers in June 1973 prior to Regents examination 
week. Except for French, the tests were scored commercially, 
and the scores were reported to the Bureau of Elementary and 
Secondary Educational Testing. The French test was hand- 
scored in the Bureau. After the Regents examinations had been 
administered and scored locally, these scores were also for- 
warded to the Bureau where they were paired with the standard- 
ized test scores. Table 12 indicates the number of pairs of 
scores which were used in this study and the schools in which 
the students were tested. 
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Table 12 

Description of the Reference Groups 



Number of Pairs of Scores 



School 

Bethlehem, Delmar 
Binghamton Central 
Cazenovia 
Dobbs Ferry 
Far Rockaway, N. Y.C. 
John E. Glenn, Huntington 
Monroe Woodbury 
Oceanside 

Total 



Biology English French Geometry Soc^ 



108 
76 

223 
176 

583 



94 



166 



260 



85 



124 



128 
337 



160 
82 



172 
203 

617 



121 
4( 



188 

35S 



The equipercentilc method was used to determine 
Comparability: scores having the same percentile rank 
were considered comparable. Three 
points in each distribution of Regents scores and one 
point in each distribution of standardized test scores 
were selected for study. They were (1) the Regents 
examination score of 90, which is considered a high 
score in each Regents examination; (2) the mean Regents 
examination score obtained in June 1972; (3) the Regents 
examination score of 65, which is the minimum passing 
score; and (4) the mean standardized test score obtained 
at the time of standardization. 

To answer the second question, the pairs of scores 
used to establish comparability were also used to compute 
Pearson product moment correlations. 



c* Results of the Study 



Table 13 shows, for each subject, the perccMitile ranks of 
selected Regents examination scores in the two sets of 
norms. The national percentile, ranks are approximations 
based upon the use of comparable scores. It can be seen 
that for all but three pairs of scores the percentage of students 
with lower scores is greater for the national norms sample 
than it is for New York State students. This indicates that 
the achievement of New York State students in June 1972 
was generally superior to that of students nationwide. The 
percentile ranks of the remaining three scores suggests 
that the achievement of New York State students at these_ 
levels in social studies a nd geometry was a pproxi mately 
equal to the achievement of students nationwid e > 

Table 13 

Percentile Ranks of Selected Regents Examination Scores 
in State and National Norms 



Percentile Rank 



Regents 


Biology 


English 


French 


Geonieti'y 


Soc. 


Stud. 


Score 


St. 


Nat. 


St. 


Nat. 


St_. 


Nat. 


St. 


Nat. 


St. 


Nat. 


90 


80 


9J 


90 


96 


90 


96 


80 


80 


85 


84 


Mean 


45 


55 


50 


74 


4 5 


67 


45 


56 


50 


50 


65 


24 


31 


14 


36 


17 


40 


28 


40 


13 


18 



A similar picture is presented in Table 14, which 
approximates the Regents examination score of tlie ''average 
student" nationwide in each subject and compare? this 
score with the mean score attained by the sample of stu- 
dents used to compute the June 1972 percentile equivalents. 
In each subject the mean score attained by students in the 
New York State norms sample exceeds the estimated mean 
score of students in the national norms sample. Ignoring 
percentile rank differences of less than fifteen paints, one 
would conclude that s tudents in New York State achieved 
at a higher level than students nationwide in En gl is h and 
Fren ch, and at the sam e level in biology, gc ometry , and 
social studies. 
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Table 14 



Mean Regents Examination Scores of State 
and National Noi'ms Samples and 
Percentile Rank in State Norms 



Svibjec t 



State Norms Sample 
Mean %ile Rank 



National Norriis Sample 
Mean %ile Rank 



Geometry 
Social Studies 



Biology 
PJnglish 
F rench 



75 
75 
74 
72 
76 



45 
50 
45 
45 
50 



74 
66 
68 
70 
74 



45 
20 
30 
40 
45 



The correlations between Regents examination and 
standardized test scores are given in Table 15. This 
table indicates that there is a substantial correlation 
between scores obtained for each pair of tests, ranging 
from . 67 to . 77. About half of the variance of one test 
is predictable from the variance of the other. 



Table 15 



Correlations Between Regents Examination 
and Standardized Test Scores 



Subject 



r 



Biology 
English 
French 



. 77 
.71 
. 74 
. 72 
. 67 



Geometry 
Social Studies 



ERIC 



In addressing the qvieslion as to wliether the Regents 
examinations should be replaced by comtne rciallv -published 
standardi^ed tests an ancilysis was undei taken to dc termine 
the advantages and disadvantages of eacli test. The follow- 
ing factors were considered: 

1) Regents examinations are more closely related to 
the properties described in Chapter lll-'-tho ''pro- 
jected program- -than standardized tests, 

2) Regents examinations are new each time they are 
administered, which varies from one to four Lin\os 
a year; standardized tests are kept on the market 
for 5-10 years without revision, 

3) Regents exams are entirely secure; standardised tests 
are readily available to classroom teachers, 

4} Regents exams are based upon State - recununoudod 
courses of study; standardized tests are based on an 
analysis of textbooks which generally arc con)j)osites 
of several courses of study, 

5) Regents exams eire 3 -hours in length; ataadardi/,ed 
tests require about 40 minutes. {In F rench, f(;r 
example, the Regents examination measvires reading, 
writing, and listening skills, but the standai dized 
test that the Department used measures onK^ reading 
comprehension* ) 

6) The Regents examination costs are much lower th^m 
those from publishers. The Regents examination costs 
are based upon an average test development expenditure 
of $8, 000 per examination and printing costs of $30 

per thousand. The Regents examinations in four of the 
five subjects measured in the study (all except I'rench) 
are among those used most widely and therefore those 
with the lowest per pupil costs* June Regents exam- 
inations < re written by from about 2, 500 pupils in 
such subjects as Italian and Hebrew to 35,000 in 
Earth Science and in Physics to over ]40,000 in 
English, in Social Studies and in Biology, The average 
overall per pupil cost for June Regents examinutionb 
is in the $• 25 to $. 35 range and would be slightly 
higher for a total January, June and August examina- 
tion program package. 
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The per pupil cost figures listed below are for 
testing the 000 pupils for this study. 



Per Pupil Cost 



Subject 



Regents Exam Standardized Test 



English 
Social Studies 
Biology 
Geometry 
French 



$. 10 
. 10 
.10 
.11 
.32 



$. 73 
. 73 
. 51 
.46 
. 32 



These figures illustrate how large some of the 
differences can be, but are not entirely typical be- 
cause Regents examination costs are directly re- 
lated to the number of tests written for each exam- 
ination because the larger the number of tests 
written the lower is the per pupil examination cost. 

7) The Department has not surveyed the usage of commer- 
cial tests in the field but based on its field experience 
is convinced that few school districts presently admin- 
ister such tests. If a decision were made to replace 
the Regents examination, either the local school district 
or the State would have to allocate additional funds. 

It is the Department's belief that the advantages of con- 
tinuing the Regents examinations clearly outweigh the dis- 
advantages until some other suitable substitute is available 
at a reasonable cost. 

Cautions in Interpreting the Results 

Studies seeking to evaluate the effectiveness of New York 
State secondary schools usually include an attempt to 
compare the achievement of students in New York State 
with the achievement of students in other states. Valid 
comparisons of this type are extremely difficult to obtain 
because of the number of variables that must be controlled. 
If, for example, a state that is especially successful in 
keeping its students in high school until graduation is com- 
pared, on the basis of achievement test scores, with a state 
having a high percentage of "dropouts", the state with many 
dropouts will be given a decided advantage because the 
students remaining generally are more successful. A similar 
advantage will accrue to a state that has, for one reason 



-44- 



or another, a large proportion of very able or highly 
motivated students. The problems encountered in 
interstate comparisons are analogous to those involved 
in comparing individual schools solely on the basis of 
achievement test scores. Nevertheless, if the variables 
which affect achievement test scores are assumed to be 
approximately equal for all states, it is possible to make 
some generalizations concerning pupil achievemei^t in 
New York State vis-a-vis pupil achievement in the rest 
of tlie nation. Indeed, such generalizations are not un- 
common. 

The major finding of this study appears to be con- 
sistent with the findings of previous studies and with 
the evidence derived from national testing programs: 
New York State students achieve at a level which is at 
least equal to or, in some subjects, is superior to that 
of students nationwide. However, the absence of norma- 
tive data for the June 1973 Regents examinations dictates 
caution in the acceptance of this conclusion. Although 
the technical characteristics of Regents examinations 
remain fairly stable from year to year, there are occa- 
sional exceptions* When the 1973 norms become avail- 
able, this study can be revised to permit the placing of 
considerably greater confidence in the results. 

It should also be noted that the comparability of 
scores established in this study is somewhat dependent 
upon the reference groups used, although comparability 
will hold reasonably well for other groups drawn from 
the same population. Table 16 shows the extent to which 
the reference groups are representative of the 1972 norms 
samples. 

The substantial correlations found between scores on 
Regents examinatioriS and scores on standardized tests do 
not, of course, suggest that the tests are interchangeable, 
Otis IQ scores correlate . 71 with Nelson Biology Test 
scores (Nelson, 1965, p. 13), . 77 with Stanford English 
scores, and . 76 with Stanford Social Studies scores 
(Gardner, 1965, p. 21), but the use of the Otis IQ test 
in place of any or all of these achievement tests would 
not be considered. The first criterion in selecting a test 
must be content validity, and no high school achievement 
test can be used in New York State unless it is based upon 
the State-recommended course of study. 
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Table 16 

Regents Examination Means and Standard Deviations 
for Reference Groups and for 1972 Norms Samples 



Reference Group 1972 Norms Sample 



Subject 


n 


Mean 


S. D. 


n 


Mean 


S. D. 


Biology 


583 


79. 7 


12. 3 


20,428 


74. 6 


15. 5 


English 


260 


72. 9 


11.7 


21, 992 


74. 5 


1 1. 5 


French 


337 


76. 8 


12. 7 


5, 348 


74. 5 


12. 9 


Geometry 


617 


75. 5 


17. 9 


20, 351 


71.6 


19. 3 


Social Studies 


355 


78. 6 


10. 8 


18,874 


75. 8 


12. 7 



0 e. Summary of the Study 

In June 1973, samples of New York State high school students 
preparing to take Regents examinations in biology, English, 
French, geometry, and social studies were administered 
commercially-published standardl^ed tests in these subjects. 
The results were used to determine the comparability of 
scores on each pair of tests, permitting the achievement of 
pupils in New York State to be compared with the achievement 
of pupils nationwide, Correlations between Regents exam- 
ination and standardized test scores were also computed. 

The results indicate that New York State students achieve 
at a level which is at least equal to or, in some subjects, is 
superior to th^t of students nationwide. Substantial correla- 
tions were found between the scores obtained on Regents 
examinations and standardized tests, ranging from ,67 to . 77, 

The Department has concluded that the Regents examina- 
tions should be retained until better instruments are available 
at a reasonable cost. 
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New York City Protot^e Study of Urban Education and 
Elementary and Secondary Education (E, S. E. A, ) - 
Title I Programs - School Year 1973-74 

a, Pjarposes of the Study 



In the process of determining what work activities the Depart- 
ment should undertake in the school year 1973-74 a review 
of past practices was undertaken. It was determined that 
four general problem areas beyond those "state of the art" 
issues discussed in Chapter III of this report could and 
should be dealt with, realizing, however, that the limitation 
of funds available would prohibit the Department from under- 
taking any effort in all school districts receiving compen- 
satory education funds. 

The first problem identified was that tests given to 
measure the effect of programs are not comparable across 
districts or across projects either in terms of the test given 
or in the time when they are administered to students. Second, 
because of high levels of student mobility and the lack of 
computer ability at the local level it is generally not feasible 
to systematically trace pre and post test scores of individual 
students by project. Third, it was determined that the effects 
of compensatory projects are difficult to separate from the 
effects of the regular educational program available to all 
students because of the absence of comparable control groups* 
Lastly, it was determined that more detailed information was 
needed regarding the projects, if more detailed analysis were 
to be performed. 

After considering these problems the Department con- 
cluded that a prototype study should be undertaken in New York 
City, since a majority of the compensatory funds are targeted 
there. This study was designed to test the feasibility of 
developing and implementing a Department (as opposed to 
district) managed evaluation program, The evaluation was 
also designed to further the development of the "projected 
program" described in Chapter III and to overcome the problems 
identified above, 
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Specifically, the purposes of this year's work activi- 
ties in this prototype activity as they relate to Question #1 
are to answer the following specific questions (the purposes 
related to question #2 are reported on pages 92-99). 

1) Are the 10 conmnaunity districts qualified to be 

E. S. E. A. - Title I and Urban Education Districts 
if a standardized test is used as an indicator of 
educational need? 

2) Regarding the distribution of qualified and unqualified 
students in the 10 community districts; 

Are all qualified students in either an 
E*S,E. A. - Title I or an Urban Education 
project? 

Are unqualified students in these programs, 
and, if so, is it at the expense of qualified 
students? 

3) Regarding differences among qualified students: 

Are some qualified students more in need of 
compeiAsatory aid than others? 

Is systematic preference in targeting E.S.E. A. 
- Title I or Urban Education programs given to 
the "most qualified" group or to the "qualified'' 
group (difference between '^most qualified" and 
"qualified" follows), or are selection decisions 
apparently random? 

Procedures used in the Study 

The data used below results from the administration of a 
standardized test of reading (the California Test Bureau/ 
McGxaw-Hill Comprehensive Test of Basic Skills - Form Q, 
Level 1) to all grade five students in 10 New York City 
community districts. Grade five was selected since the 
Department's Reading Bureau concluded that results of 
any grade selected below that level might be overly affected 
by the reading approach used (i.e. if skills are taught in 
different sequences in various approaches, the test se- 
lected might or might not match that sequencing). The 



CTBS testNvaa selected because it provides historical data 
on what students nationwi^^c ^ ve done on a pre and post 
test basis. Operationally, Uu3 means that after the post 
test is administered in the spring o£ 1974 it will be possible 
to determine whether students in compensatory projects 
have made greater gains in reading achievement than would 
be predicted from national performance data. (This will be 
reported in the fall of 1974. ) A computerized student file 
has been established to insure that the post-tests are 
properly matched to the pre-tests. In addition, detailed 
information on the project has been collected. The types 
of program data and their use will be reviewed under 
Question /^2/ 

An additional element of great interest to the participating 
districts will be included in a fall report. As indicated in 
Chapter II, the Department is in the process of developing 
a "reading effectiveness measure'^ In the fall of 1973, 
data were collected on what reading and social studies in- 
structional materials are used in each classroom. The 
readability of these materials is presently being analyzed 
using existing readability formulas* In the spring of 1974 
an "experimental version" of the reading effectiveness 
measure will be administered. In this experimental 
version, test item difficulty will be controlled and a 
quasi-cloze technique used for deleting words in passages 
which are ranked according to difficulty. As a result of 
this activity, it will be possible to determine whether or 
not students have met the reading "standards" set by their 
reading and social study materials, and to determine 
whether or not the grade -level difficulties of the materials 
correspond to the grade level reading ability of the students. 

Results from Data Presently Available 

For the first time, the data available for analysis in this 
ten district san.ple provide an unduplicated count of fiftK 
graders, categorized on the basis of numbers enrolled in 
Title I programs. Urban Education programs, both programs 
or neither program. It also provides a distribution of these 
students' pretest scores on the reading comprehension 
section of the California Test Bureau/McGraw-Hill Com- 
prehensive Test of Basic Skills - Form Q, Level I, The 
distribution, or percentile ranking, of scores was related 
to the national norms for this test and was used to de- 
termine how students taking this test do relative to one 
another, and to the original group of students npon whom 
the norms are based. 
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While interpretation of the enrollment by program 
data is fairly straightforward, interpretation of tho test 
score data is considerably more difficult. Methodological 
difficulties present in standardized tests cloud any inter- 
pretation of test scores or petcentile rankings --beyond 
comparison of students to each other or to the norm group. 
Yet other questions which one would hope testing instruments 
would address are clearly more fundamentally important. 

For example^ suppose the average score for a school 
district on this test is the same as the average score ob- 
tained by all students in the norming group, thereby 
placing the school at the 50th percentile. Can we say that, 
on the average, this school district has fulfilled its re- 
sponsibility to teach students all that they should know at 
this point in their education? Clearly we cannot. All we 
know is that these students, on the average, did as well 
as the norming group. What we do not know is whether 
what the norming group did on the average was enough to 
warrant determination that their education was in some 
fundamental sense adequate. The answer to this question 
could only be provided by a criterion-referenced test, 
which had specified levels of competence by grade level, 
and which was systematical ly related to some well-defined 
'^anchor" statement of skills necessary for the effective 
functioning of graduates in society. 

Unfortunately, the state of the art in testing is not 
sufficiently advanced at this point to make usage of such 
tests practical. Hence, this analysis is forced to make 
the best interpretation possible from existing standardize'd 
test results and to accept the ambiguities in interpretation 
as are unavoidable. 

In this analysis, a major area of ambiguity which 
cannot be avoided involves the determination of what con-, 
stitutes a "qualified" as opposed'^to "unqualified" student 
for Title I and for Urban Education classification purposes. 
This is a critical determination because much of the po- 
tential usefulness of this type of data lies in the direction 
of being able to determine accurately the numbers of stu- 
dents being served by Title I and Urban Education programs 
relative to the total need for these programs. 



Since no criterion-based instrument is available to 
determine how many students are failing to achieve at 
logical levels of competence for their age/grade level, 
we have to fall back on definitions of "qualified" and 
'^unqualified" which are based on each student's standing 
relative to his peers and/or to the norm group. This 
ambiguity enters when students are designated as quali- 
fied solely because they achieve at levels below other 
students. It may be that in fact both groups of students 
(or neither group) are achieving adequately in terms of 
a steady progression toward reasonable adult compe- 
tencies* This we do not know and, as a result, educators 
are forced to make some arbitrary classification based 
on relative percentile ratings of various groups of students. 

Convention dictates that any student achieving below 
"grade level" {i.e, below the 50th percentile) be viewed 
as falling behind and warranting special attention. The 
Department will adopt that arbitrary convention and 
augment it somewhat by dividing this group into two parts; 
the first ('most qualified") and second ("qualified") 
quarters. The first quarter contains students in the 1-25 
percentile rank range, and the second quarter those in 
the 26-50 percentile rank range. All students achievirtg 
above the 50th percentile will be taken to be "unqualified" 
or inappropriate targets of Title I - Urban Education 
programs. 

Another and similarly arbitrary definition of "qualified" 
students includes those students achieving more than one 
year behind "grade level", i.e,, below approximately the 
33rd percentile on the California Test Bureau/McGraw-Hill 
teat. This definition was rejected mainly because the avail- 
able data are not categorized on this basis, and exposition 
would necessarily be more involved to accommodate this 
problem without any real additional benefits to be gained* 

The following analysis will center on addressing 
several questions of an evaluative nature. The answers 
to these questions should provide interesting insights into 
the role of compensatory programs in the 10 districts in 
this sample. The questions to be addressed are as follows: 
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1) Are the 10 districts qualified to be Title I and 
Urban Education districts? 

2) The distribution of qualified and unqualified 
students in the 10 district sample: 

a) Are all qualified students in either a Title I 
or Urban Education program? 

b) Are unqualified students in these programs, 
and if so, is it at the expense of qualified 
students? 



3) Differences among qualified students* 

a) Are some students more qualified than others 
to be the targets of compensatory programs? 

b) Is systematic preference in targeting Title I 
or Urban Education programs given to the 
mos t qualified group or the qualified group, 

or are selection decisions apparently random? 

These questions will be addressed in the analysis . 
which follows. 



(1) Are the Ten Districts Qualified to be 
Title I and Urban Education Districts? 

The data indicate that 72 percent, or 
17,403 of the 24, 172 fifth graders in the 
ten district sample, were scoring at the 
50th pe rcentile'»< or below on the Cali- 
fornia Test Bureau/McGraw-Hill Com- 
prehensive Test of Basic Skills. This 
indicates that 22% more of the fifth graders 
in these districts were achieving below 
grade level (i.e., below the 50th percentile) , 
than would be expected if achievement 



Achievement at the 50th percentile means a grade equivalent score of 5. 0 on 
this test. However, this test was given in November almost three months after 
the start of the school year. In this sense, grade level should be about 5. 3, or 
above the 50th percentile. Hence the inclusion of 50th percentile students in 
the qualified group. 




levels in tKese (i islt'icUj t ^}^l(-^h cuIJJhi 
national achicven ien tj\ c>r ni s o o cUl o d 
for this test . On the a.s ijiirnplion that 
the minimally desired adnevejiu nt U?vol 
for these students is the average achieve- 
ment level for the norminij ^^roup, clearly 
the ten districts sampled qvialify in tlie 
aggregate as having exceptionally large 
nun^bers of underaclnevc rs. 

On a district- by-district basis, how- 
ever, the variation in tlie i:^ ^' 1] ^V\J^L 
students achievi nr; at or^bob)v>^ J^^Illl 
percentile rang es /roni^a !<i\\ ci 6 
percent to a hif;h of 8L o jific^^iit, \\':lh 
all but one dist rict liavuM^ rnoi < aa'^ 
60 percent of its stud onl;.'. in inU: ^'-■-^vv^y* 

There appear to he two inojijc facU-^rs 
which can account for a district clotse to 
national norm.'a in achievement l)ecomiiig 
a recipient of Urban Education and Title I 
funds. First, eligibility i determined 
in part on the basis oi inrlividual ;;c]>ool building 
needs, not just on the school district's 
overall situation. Hence, a district 
which, overall, may i^e acliieving at 
national norms niigl^t still be eligible if 
one school in tlie di^Urict ^^a^J in clear need 
of compensatory retjources. Secondly, 
compensatory aid traditionally has been 
allocated on the basis of both ujide racJiieve - 
ment and poverty, raUier than just vuider* 
achievement alone. 

The Distribution of C3vuilified tncl Unquali- 
fied Students in the Teji Di t^ti-i^U Sample 

(a) Are all qualified slacKrulij in eitlior 
E.S.E.A, Title T (;r Url^au i'jduca- 
tion programs? 

As was noted al)ove, 7?, percent or 
17,403 of tlie ZA, ]7?. fiflh »^rad^'rs 
in the ten district .saraple appear 
to be qualified to i)cncfU. from 



Title I and llrlKvn i'lducalion pro- 
grams. Jiinrollmeul in Title I 
and Urban Education programs 
is 6,393 or ortly 37 percent of 
the total cn.ialificd .student popula- 
tion in the Un\ districts. Jlowever, 
this overstates ihe extent to which 
the need in lhe:3e districts are be- 
ing addresiied since sorne of these 
students in tliese progratiis appear 
unqualified for them on the basis 
of achievement. Subtracting out 
the students in tlie.se pro^^rams who 
are achieving at levels above tlae 
50th percentile leav(}s only S,716 
qualified students beii'ig served. 
This figure repveGcnts only 33 
percent of the total nvniiber of 
qualified student;-;. Overall, then, 
^ f*-'!! two -thirds of thos e students 
likely to bene fit f roni Urban Edu- 
cation and / V J r T [ tie p rograms 
are not being ^*'£'lX5^i_hy e ither of 
these pro grams , 

In terms of individual districts 
in the ten district samj^le, thore 
is a variation of between 11,3 
percent and 6 3. 3 percent in Llie 
qualified students Ijeing served 
by these programs, depejuling 
on whicli district is being reviewed* 
This suggests tluit between 34, 7 
percent and 88. 7 percen t of the 
total need^or the 1 j vh a n Education 
and E . S. E> A, -jritl^c I programs 
at tlie fiftl i J j^r. u k lii:^: L 1 c urrently 
unmet, depend i ri :;_o n _Ui e ^li_p t r i c t 
being coj}> s i (Uj red. 



-54- 



(b) Are unqualifiod imicK'iUj^ in eil1\er 
the Urban ]^:duc:ati(>)i Js.S.ls.A. 
Title 1 prograins, aiul if is it 
at the expense of cpialificfl .;lu(tenls? 

If unqualified sUuloinrj arv^ cii;nruirl 
as those pre Ic^ Ling al or aloove 
the 50th percentile, answer is; 

yes, there are ur»qual j :'i cd siudojits 
in the Urbaii r>huNi I ioi ^uid Tide ! 
programs in llu^ iiist*;ici: ruun.Dle* 
6 78 fift h -^r ra (1 cjy^ U I \^ i a ) f h i :^ 
category . If (hoy ro[<^.u:od 
by qualified jHu'!iM-iU^ -'U * ^Sa! 
number of quaVifu^^! si;;.:.:n! > hvi-nu 
served would increa:;u 0, > ^ 
or 37 percent cH' tlie ic^la! pool of 
qualified studenl^^. Tins \^ Oiild 
mean that an a<l^ii iTiti:; 1 -1 pei^Ckjnl 
of the qualified .si^ ici ir! v.ould \)c 
served by lliese j)rnL' rains. 

This f i u u r o c a n 1 ; e v i i : \\ < i d from 
another, o r c; [ < . s i i ) v o p < - 1* p e f.' ti v e : 
tliat district cUoH > l > i?),>?rre lliat 
available plac^^^ in i i h iii ImIucci- 
tion and J itlo f j>r>'/;. i .>ni,^ ar<j fjlled 
by studenlb tlial the ;;rui;ranis were 
intended to s e r v c i j d v t I ^ n < j i ) e r 
cent successful. i i ( i . ^• , e r , u v e n 
if the districts hnd Ijceii 100 percent 
successful ill liu ^5 re;;a rd, the inpe- 
c i a I needs of a UxW (} ^ per c e n I , or 
almost tu o-thirchi of iM fiflii |;r iders 
achieving below grade norms w ould 
remain unmet by Urban I' : due a lion 
and Title [ prci^^raro^: ir. the ten 
districts. 

The numl^e r of unnii:iKfii:<l In- 
dents in individoal dis^i'icVs of the 
ten district S/UiM>!e \ <; /■i'i};;}d- 
erably. Vicwc^d a ^ jwoivuMion 
of the total nin:tV;er ('1 qwaiiTied 
students in c:ie'; r ^ ^ . M hi:n.!je>- 
of unqualified iJiudei to iis f itle f 
and Urban Kducafirin lo ^.r^ons 



varies from 0. 5 percent to 9. 9 
percent of this total, depending 
on which district is being examined. 
This, of course, suggests that an 
additional 0. 5-9. 9 percent of the 
students ciu alified for these pro- 
grams in individ ual <^UHtr_icts 
could b e ser ved ))y th em if these 
districts were totally successfu l 
in targeting only on the aj:?pro> 
priate stud ents for th ese prop;rams . 

(3) Differences Among Qualified Students 

(a) Are some students more qualified 
than others for entry into Urban 
Education and E. S. E. A. Title I 
programs by virtue of their edu- 
cational deficiencicjs ? 

Of the 7Z percent of the students in 
the ten district simple who scored 
at or beJow the ^jOlh percentile on 
the pretest, and hence warranted 
a designation c.s likely to benefit 
from Urban Education and/or 
Title I programs, 42 percent 
scored i n tVi e bo ttom q uarter (1- 2 5 
pe rcen t^le ran gc) > The remaining 
' • ^ 0 percen t scored in the second 
q^uarter (26 - 50 pe rce ntile rang e ) . 
Clearly those 42 pe rcen t achievi ng 
in the bottom quarter have a more 
clearly demonstrated neni I'or 
compensatory pi^ojj^i-an^ assistance, 
and from this perspective are n\ore 
qualified for com])ensa tory programs 
than the 30 percent of students in 
the second quarter. For purposes 
of further analysis we will desig- 
nate the students in the bottom 
quarter as "n^ost qualified" and 
the students in the second quarter 
as simply "qualifiecr'. 



Is , systematic preference for 
admittance to F:.S,R,A, Title I 
or Urban Education programs 
given to students in the "quaU- 
fied" group or the "most quali- 
fied^' group, or do preference 
decisions appear to be random? 

Establishing prioritie.s among 
different groups that stand to 
benefit from exposure to com- 
pensatory education programs 
is a complicated problem. 
Should the limited resources 
available be concentrated on 
those most qualified for assist- 
ance because they have the 
greatest need? Or sl^ould they 
be concentrateci among students 
in the 26 -SO perccM)li!e ran^e 
on the assiuiipti Oil that this 
group, because their problen\s 
are more limited, are more in 
a position lo benefit from com- 
pensatory programs? VVliichever 
way this decision is niurle, it 
would clearly l)e vuireasonable to 
orient compensatory programs 
toward students wlu) are achieving 
above nornis for tlieir age/grade 
group. 

Table 17 provides an overall 
picture of how students were se- 
lected for Title I and Urban Edu- 
cation programs in t)ie ten district 
sample. In Table 25, data are 
presented to sliow the proportions 
of ^^most qualified", 'qualified" 
and ''unqualified" students in the 
total 5th grade enroUment, the 
Urban Education program enroll- 
ment, the Title I progran\ enroll- 
ment, and the enrollment in both 
Urban Educatiun ancl Title I pro- 
grams. 



rela t. wcj.y J^^'y^y^p^^* t^"^ t ^ vom the 
ugq i>a lif i '^^^^^ J-'yj\J^'e_iil_the^ 

The y nun/tbor Oj?S (uy about 10.6 
percent o/ tluj lou! Miu^iber of 
s tucleul a in ili i-^i £ a t ory 
pro gvix m G , learly th es e figures 
i ndicaic tluU, the re iio con certed 
effor t lo fill tho:i=e provj;.Vc vms w ith 

unqvial] f i cd ^ tud e n t s , \ or_ do these 

number s y u itjief \^\^\ a I unc ualified 
sludents <^^^^cye}\^ beir : vole c ted 

rand cnnjy- ilJpJ ^ J^"^ ^ ' ^ ' - '■ ' ^' ^U^^HtS. 
qualified ^lndr^nl s^ 

Comparison liinoiu; I he most 
qu£ilified ^-iTMup ajid Llie qualified 
group, however, does ncit iii 
general yield hiulk dear -cut 
results, excvifi i<a- the .small 
group of stuficriils in botji IJrlKin 
Education dnd lillo ! programs. 

^ilI.L^^J?.i_2I! 4 ^ ' ' ^J' ^1 5 tudonts a re 
from I he ni o si c|u jjb e d_ ij^ r o i ij), 
indie a ti n c ] c a r refer ence 
patte rn in Jtcrms of student se > 
lec ti on fen* ih i s j oi n t p r 
eff oi' U file results? for students 
in ^Hhe^r tirl^arj V.dr.caUon or Fitle I 
programs arc tiot as clear. With 
70.0 pcrcejiL nl] Uvbau I^ducation 
students <intl o I, norcetit of all 
Title J slud.:nl r-.^min^.; from the 
most qucL}n"it;d ;Urtila, clearly the 
general emr^liasj^; iei^cis to l>c on 
thjs lype of iM.urlonL, Ifowever, 
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the fact tliat a full 21, 7 percent 
of the Urban Education and 2 3, 8 
percent of the Title I students are 
from the (less educationally dis» 
advantaged), qualified strata indi^ 
cates there probably is no clear^ 
cut preference pattern between 
these groups used in targeting 
Urban Education and Title I pro- 
grams. 

Ranking these programs, the 
process for selecting students to 
be in both Urban Education and 
Title I programs placed heaviest 
emphasis (88.4 percent) on selec- 
tion of students from the most 
qualified group. Urban Education 
was next with 70 percent of the 
enrollment being selected from 
students in the most qualified 
category. Title I fell last with 
only 64. 3 percent of the program ^ 
enrollment falling in the most 
qualified category. Title I also 
had the largest percentages of 
both qualified (23. 8 percent) and 
unqualified (11.9 percent) students 
in the program. 

If the targeting process was 
totally oriented toward selection 
from only the most qualified group 
of students, 2, 134 more places 
in Urban Education and Title I 
programs would have been avail- 
able for students in the most 
qualified group. This would have 
meant that 36. 4% of the most 
qualified students now in neither 
of these programs could have been 
served in one of them. 
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In summary, the analysis pre- 
sented above and that to follow in 
the spring are significant improve- 
ments over what has been done in 
the past. In addition, this proto- 
type will allow similar improve - 
^ ments under Question #2 reported 

on pages 

Evaluation of Urban Education Projects - School Year 1972- 73 

a. Purposes of Evaluation 

Since the Governor and Legislature have a special interest in 
the State's Urban Education Program, this section of the 
overall report includes more descriptive information than 
other sections. One of the purposes, then, of this section 
is to review: why the Urban Education Program was estab- 
lished, why the project approachwas adopted, and how projects 
are initiated^ approved, financed and amended* In addition, 
an overall profile of projects and a report on how the re- 
sources are distributed by program type are presented. 
Lastly, the success of urban education projects is reviewed. 
This last component addresses itself to question //l--to what 
extent are New York State's elementary and secondary stu-- 
dents successfully mastering basic skills? Specific exem- 
plary projects are included under question ^2 on pages 

b. Review of the Need for Urban Education 

Educational problems that result strictly from poor education 
usually can be cured with improved education. However, 
educational problems that result from related circumstances 
such as poverty, poor housing, employment, or pervasive 
anxiety abou; matters involving money or lack of it may 
not always be solved with improvements in education. Thus 
those problems which stem from society's general need for 
food, clothing or health may not be completely solved through 
education in general or the Urban Education Program speci- 
fically. 

The major intent of the Urban Education legislation was 
to alleviate educational problems associated witli poverty* 
From the view of the educator, one problem stood above all 
others: in 1967 when the Urban Education Program was 
fundedjStudents, and especially those with a background of 
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poverty, were falling furtlier behind grade level with the 
passing of each year* It became obvious that efforts were 
required to reverse this trend if meaningful progress toward 
equalizing educational opportunity was to be made. The 
Urban Education Program was an attempt to meet this need. 

Estimates of the number of students participating in 
Urban Education programs provide some indication of the 
need for compensatory programs such as Urban Education, 
During the 1972-73 academic year there were over 500,000 
duplicated participants in the Urban Education program. The 
number of duplicated participants is detailed elsewhere in 
the report. However, it should be noted that this figure re- 
presents but a small portion of the total number of students 
with poverty-related education deficiencies. It is presented 
mainly to show that the magnitude of the need for compensa- 
tory education is substantial. It should be noted that partici- 
pants in Urban Education projects range in age from the pre- 
kindergarten child to the unemployed or underemployed adult 
who has not received a high school diploma or passed the 
high school equivalency test. 

The Department has never, in the past, had firm figures 
from each district relative to the unduplicated count of parti- 
cipants in the Urban Education program or other categorically 
aided programs. In that regard, a form for eliciting such 
data from school districts for use with the 1973-74 report 
has been devised. Not only will such a report deal with the 
unduplicated count of participants across projects within a 
given program (e. g. the State Urban Education Program) 
but also across programs (E. S. E. A. - Title I and Urban 
Education) within a given district. The indicators from the 
preliminary 1973-74 data which were reviewed in the previous 
section strongly support the contention that Urban Education 
programs are serving only a small portion of those students 
who could stand to benefit from them. 

A number of alternative strategies for addressing the 
problem of Urban Education have been considered over the 
years. New forms of school financing were recommended 
as a means of solving the problems as were adjustments in 
local control, integration, busing, and compensatory educa- 
tion. While the preceding alternatives all have potential for 
addressing certain urban problems, the designers of the 
Urban Education Program felt that compensatory education 
was especially promising as a strategy for equalizing educa- 
tional opportunity. 



Stated succinctly, if the usual exposure to an educational 
experience did not bring about the results desired by society, 
it was hypothesized that an additional exposure would compen- 
sate or put the learner back on grade level. Thus compensa- 
tory education was a suggested technique at both the national 
and State levels. 

In its broadest outlook compensatory education was de- 
signed to supplement ^ not supplant , facets of ongoing educa- 
tion. In New York State, for example, learners who ;ire not 
up to grade level continue to be taught the regular reading 
material that is paid for by general aid. Moreover, such 
learners are taught an additional set of lessons paid for by 
compensatory aid. Thus, the exposure to reading for some 
pupils is specialized, more concentrated, and provided for 
a longer period of time. Devising a means for offering such 
supplemental work has centered in the "project" concept. 
The word project as used here means a planned act ion in- 
volving the learning of a phase of school work resulting in 
the general achievement by the student of some specifically 
stated objectives. 

In the 1972-73 academic year, there were 218 funded 
projects in the Urban Education Program in New York State. 
While a detailed breakdown of the total expenditures will be 
presented later in this section, it is important to know at 
this point that the're were forty-seven million dollars^ avail- 
able for projects in the Urban Education Program. The re- 
mainder of the report will concern itself with the specifics 
of such projects, 

c. Procedures Used in the Urban Education Program - 

How Urban Education Projects are Initiated, Approved, 
Financed and Amended 

Urban Education funds are allocated by the Legislature on an 
annual basis. They are then divided among urban school 
districts in New York State on the basis of a predetermined 
formula. This formula establishes the maximum grant 
available for an individual district. 



Less $ 180, 000 ($90, 000 each) in the form of a direct grant to Roosevelt 
and Wyaildanch, rather than in the form of reimbursement for approved 
expenditures, 



After a district is notified of its Urban Education 
maximum grant allocation and the current guidelines 
and priorities of the Urban Education program, it then 
submits a project proposal or several project proposals 
for reimbursement of approved local expenditures. The 
district has tlie option of submitting a proposal for a 
completely new project or for the continuation of the pre- 
vious year's project. In theory, project^which meet de- 
sirable objectives are resubmitted in subsequent years. 
Additionally, project?', which don't quite meet their ob- 
jectives are improved through changes in methodology or 
objectives and are also resubmitted. 

Project proposals submitted by local school districts 
to the Urban Edvication Office fall into two categories, 
new and resubmitted. 

New Project Proposals are reviewed by the professional 
staff member responsible for the district submitting the 
proposal. The Urban Education staff then determine which 
other units within the Department should review the proposal. 
The proposal is then routed to the designated subject area 
specialists for review of the educational approach and to the 
evaluation unit for review of the proposed evaluation design. 

Each unit involved recommends approval or disapproval 
of the proposed expenditures to the professional staff member 
in the Urban Education office. The Urban Education staff 
member then coordinates the various reviews. The final 
decision for approval or disapproval is made by the respon- 
sible staff member in the Urban Education office. After the 
pr oject/sproposed expenditures have been dete rmined to be 
reimbursable by the Urban Education office it is sent to the 
state -^aided programs unit in the Department for a review 
of the project budget in terms of appropriateness and accu- 
racy» The revised/approved budget is sent to the local 
district directly by the state-aided program unit. 

Resubmitted project proposals are for projects that 
have been approved and in operation in previous years. 
They are reviewed for approval only by the responsible 



professional staff member 
and the evaluation unit. 




Recycled projects need not be reviewed by subject 
area specialists. They are sent a copy of the proposal. 
They may review the project proposal but it is not re- 
quiredt 

The evaluation unit makes its recommendation for 
approval or disapproval of the recycled project's evalua- 
tion design to the ITrban Education staff member. The 
staff member then makes the final decision £or approval 
or disapproval. 

As with new projects, after the recycled project lias 
received final approval it is then sent to the state-aided 
programs unit of educational finance for review and 
approval of the budget. 

Urban Education funds allocated by the Legislature 
on an annual basis are suballocated by a predetermined 
formula which takes into consideration (1) number of 
children from families receiving AFDC, (2) number of 
sixth grade pupils scoring below level on the PEP test 
in 1967 and (3) weighted average daily attendance of the 
district. It should be noted that, since 1967 data continues 
to be used rather than more recent resulti^ each district's 
proportionate share of available funds has remained con- 
s tan t. 

Each district eligible for Urban Aid gets a proportion 
of the total allocation based on the formula. The formula 
allocation for each school district receiving Urban Education 
funds in 1972-73 is presented in Table 18. For purposes 
of comparison of the magnitude and distribution of these 
funds, the 1972-73 allocations for various other compensa- 
tory aid programs are also listed in this table. 

Amendments to project proposals fall into three cate- 
gories; program change or modification, budget change, 
or both program and budget change. 

The amendments are sent to the state-aided programs 
unit. If the modification includes both program and budget 
changes the program office reviews the modification and 
then returns the approved program modification to the state- 
aided program unit. Amendments containing budget modifi- 
cations are accepted for processing by the state-aided pro- 
grams unit in September, December and March. Only 
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one modification per project is accepted during each of 
of these periods. 

Comparative evaluation is a continuing effort to obtain 
information for making decisions. Additionally, the best 
decisions are based on cumulative results. Thus, evalua- 
tion findings can and should be used to bring about changes 
in a project's objectives or activities during the actual life 
of the project. 

Minor modifications may be made without prior con- 
sent, but major changes in program, objectives, evalua- 
tion design, or budget require such approval. 

Some types of project modifications are shown in 
Table 19. 

From the table it may be seen that a relatively small 
proportion of projects had modifications in objectives or 
activities prior l-o or after project implementation. Addi- 
tionally, tl'ie majority of such modificati ona took place 
after consultation with the State Education Department 
(SED). 

Table 19 

Urban Education Project Modifications 



Modification 


No. of 

Projects 

Indicating 


Percent of 
Total In- 
dicating 


Yes 


No 


Yes 


No 


Objectives prior 
to iniplementation 


12 


206 


5. 5 


94. 5 


Approved by 
SED 


6 


6 


50. 0 


50. 0 


Objectives after 
implementation 


5 


213 


2. 3 


97. 7 


Approved by 
SED 


5 


0 


100. 0 


0. 0 ' 


Activities prior 
to implementation 


11 


207 


5. 0 


95. 0 


Activities After 
implementation 


IZ 


196 


10. 1 


■39. 9 
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d. A Profile of Urban Education Projects 

Data in this section were collected from all Urban Education 
school districts on mailed report forms. Satisfactory in- 
formation was obtained from all districts. 

Community Involvement in Planning 

School districts are urged to elicit the help of district 
residents in establishing Urban Education programs. Such 
help may be in the form of planning, needs assessment or 
the establishment of priorities. Since sheer numbers in a 
given category do not indicate community participation, 
several sets of data were generated to obtain a broader view 
of this aspect of Urban Education programs. 

From Table 20 it may be seen that the District Advisory 
Committee was the group v/ith the largest number of partici- 
pants and also was the type of group most frequently used by 
school districts for project planning. Approximately 80 per- 
cent of the projects utilized 3 or more of these groups in 
their planning efforts. 



Table 20 

Number of Individuals Who Participated 
in Planning at the District Level 





Number 




Participants 


Projec ts 


District School Board 


1, 316 


170 


District Advisory Committee 


6, 14Z 


209 


Local PTA or Home /School Association 


5, 844 


132 


Representatives from Local Community 
Organizations 


1, 337 


151 


Nonpublic^ School Offici^als 
Other Groupra j 


499 


107 


1, 352 


72 


TOTAL 


16, 490 
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Project Time Span 

F^rojects could fall into one of three broad time cate- 
gories as shown in Table 2U 



Table 21 



Project Time Span 



Time Span 


Nvimber of Projects 


Percent of Total 


Regular School Year 


178 


81.7 % 


Summer Only 


9 


4. 1 


Calendar Year (12 months) 


31 


14.2 


Total 


218 


I (10 



p'rom tlie table it may be seen that the larger number 
of projects were those which were funded for the regular 
school year* Similarly, 4. 1 percent wore funded for the 
summer only and 14, Z percent were funded for a full 
calendar year or for a 1 2 -month period. 



Table 22 presents a more detailed analysis of project 
time span for the most common components fonnd in Urban 
Education projects. This table presents information on the 
average number of weeks and hours per week that critical 
activities in these project con^ponents take place. 



Table Zl 



Average Duration of Urban Education Offerings 



C omponent 


Activity 


Number of Weeks 


Ifours Per Week 


Activity 
Operates 


Individual 
Participants 


Activity 
Ope rates 


' Individual 
^ Participates 


Reading 
Grades 4 --6 


Diagnostic 
Remedial 


35 


3 1 


23 


8 


Reading 
Grades 1 - 3 


Diagnostic h. 
Remedial 


35 


34 


23 i 10 


Reading 
Grades 7-9 


Diagnostic & 
Remedial 


36 


31 


1 ' " — 

21 i 6 


English as a 
Second I^ng. 
Grades 10- 12 


Develop- 
mental 


38 


38 


38 


4 


Rcadirm 
Grades 7-9 


Develop- 
mental 


38 


34 

1 — — — 3 


T 

22 ! 12 



rhe table is read as follows; If a component is operated 40 weeks for three hrs, 
a day, and 6 groups of pupils are each scheduled lor 1 hour's instruction daily 
for 20 weeks, cokmms 2-5 in the table (reading from left to right) would be 40; 
20; i5; 5. 



Another aspect of project time span is the number of 
years a project has been in existence, t^ron^ Table 23 it 
may be seen that of the total projects in operation, the 
majority (b9. 2 percent) were operational for three; or 
more years , 

Table 2 3 



Years of Project Operation 





Number of 


Pe rcent of 


I Cumulative 


Years 


Projects 


Total 


i Percent 


First Year 


36 


16. 5 "A 


i 16.7% 


Second Year 
Third Year 


A 8 


24. 3 

31.2 


i 40. S 
1 ' 72,0 


Fourth Year 




18. 8 


1 90. 8 


Five or iriore 


20 


. 9. 2 


100.0 


Total , 


2 18 


^100.0 1 



To- 



ot' Uie 2tH projcclb, 6 3. 8 percent were expected to 
be rcHubinittocl in the JV73-74 academic year, ^. S percent 
w(M'e expected to be resubiiiitted, and ?3.4 percent had 
an i;iu:erlain I'utui'e according to appropriate district per- 
.sonneK 1 he Division of Urban Education had analy/.ed 
J 97?. -7 3 projects in relation to the reading, niatl\ematics, 
and bilingual education priorities to be rnaiKlated in 
1973-74. r^islricts were then advised about the potential 
for recyclin;,^ 1 97Z - 7 3 pr ojec ts in 1 973-74 in relation to the 
I) rioj'i ticy , 



JiLlV?llf^JillS:il!: V .Jl-Ll 'Ulii: Si.^* ou|) a nd P r o j c c t T' y pe 

District co(^r<)inators of Urban Kchication projects were 
rerjuested to use a standard procedure for as^jcssing student 
ethnic l^ackgr ovuuls and to provide ihc nunAber in cvich ethnic 
^;roup, who were expected to partici])ate as indicated in the 
proposal as well as the number of actual participants. 

From Tabic 24 it T7iay be seen that ihe largest number 
of participants were J3lack. Information from 23 project^J 
was classified as '^failed to designate". 



1 able Z4 
JOthnic Group 



Kthnic Group 


Pa rt id print 


T 1 

f Number 
j of 
Pro iects 


Failed' 

i^esigna I e 


to 

•xpected 


Total 
Actual 
Partic. 


Kxpect ed 


of Total 


177 


Percent 


Actual No. 
of ParLic. 


Number of 
Pro iects 


American 
Indian 


303 


0.1% 


58.4% 
87.2 


i 
1 

i 

i 

i 


4 




181 


Black 


190,979 


41.9 


166, 527 


7,592 


r 

174,119 


Oriental 


4,098 


0.9 


3,640 


( 

88.8 i 


65 


3 , 705 


Spanish 
Sur named 
American 


1 — ■ " — —\ 

201,897 

1 , 


44.3 

1 . 


165,177 


1 

81.8 ,' 


4,280 




169,457 


Other • \ 
Including 1 58,167 j 12.8 
White j ! 


52, 511 , 


90.3 


5,332 




57,843 


TOTAL 


435,444 1 '"100.0 


' j 

388,032 1 


»^-2 L/7,273 1 /' 23 


405,305 
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From Table 25 it may be seen that the anticipated parti- 
cipants in both the public and nonpublic schools exceeded the 
actual participants by 9. 7 percent in the most common com- 
ponents of Urban Education projects. Additionally, nonpublic 
school participants constituted less than one percent of the 
total actual participants for these components* Participa- 
tion of nonpublic school pupils in regular, in-school programs 
is illegal. However, as district residents nonpublic school 
pupils are eligible for off-hours, week-end, and summer 
Urban Education activities. However, throughout the State, 
the total number of nonpublic school participants in Urban 
Education projects was reported as 62. 543, the Latter 
represents 15.4 percent of the total participants in the 
218 Urban Education projects. 



Table 25 



Participants in Five Major Components 
by Type of School 









Tvpe of Participants 


Component 


Activity 


Objective 


Public School 
Participants 


NonpMblic School 
Participants 


lotal 








Expected 


Partici- 
pating 


Expected 


Partici- 
patinR 


E^;pected 


Partici- 
pating 


Reading 
Gr. 4-6 


Diagn . 
& Reme» 


Achieve • 


14,724 


13,367 


10 


8 


14,734 


13,375 


Reading 
Gr. 1-3 


Diagn. 
& Reme. 


Achieve . 


5,948 


5,716 


20 


20 


5,968 


5,736 


Reading 
Gr, 7-9 


Diagn, 
& Reme. 


Achieve , 


10,086 


9,738 


5 


5 


10,091 


9,743 


ESL 


Develop. 


Achieve. 


6,659 


5,053 






6,654 


5,053 


Reading 
Gr. 7-9 


Develop^ 


Achieve. 


7.763 


_ 7.298 






7,763 


7.?9a 
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Table 26 

A Comparison of Actual Participation by Project Component 
between the Urban Education Program 
and ESEA, Title I Program 



Urban Educ. ESEA, T itle I 



Project Component 


Program 


Program 




Actual 


Actual 


Supportive Services 


120,446 










Cul tural Enrichment 


73,592 


30,013 


Mathematics 


52,053 


118,657 


Other Basic Skills (Science, Social Studies, 






Knglish, Foreign Lang^iages) 


50,530 


30,137 


Pupil Personnel Services 


42,164 


49,436 


Inservice Education 


26,235 


11,107 


Curriculum Development 


16,724 


2,579 


English as a Second Language 


13,077 


15,491 


Health Education 


7,539 


7,710 


Bilingual Education 


6,554 


16,335 


Vocational Occupational Education 


5,652 


1,696 


Adult Career Education 


3,884 


490 


Handicapped 


1,365 


8,649 


TOTAL: 


529,853 


605 ,851 



i 

I 
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A statewide profile of the priority emphasis given to 
different aspects of education in Urban Education projects 
can be inferred from Table 26. These priorities are also 
compared with a profile of E.S.E.A. - Title I compensa- 
tory education projects in Table 26. The first survey of 
all components of Urban Education projects revealed that 
Supportive Services touched the largest number of pupils 
(120,446) in the Urban Education Program and that read- 
ing treatments involved the largest number (272,076) of 
E.S.E.A. - Title I pupils (Table 26). These data reflect 
the differences in program purposes. The Urban Educa- 
tion program in 1972-73 was continuing in supporting the 
concept of community education activities including pro- 
grams for out-of-school youth and adults. The second 
largest number of component-participants in the Urban 
Education Program were enrolled in reading activities, 
while the second largest number of component-participants 
in E.S, E. A. - Title I were in mathematics. Mathematics 
ranked fourth for Urban Education component-participants, 

A comparison of absolute numbers of component- 
participants is not especially meaningful since (I) the 
E.S* E. A. - Title I allocation is over 3^ times that of 
Urban Education, and (2) the different program offices 
use different recoinmended per pupil expenditures (not 
in a ratio of 3. 5:1), because of the difference in funding 
levels and program emphasis. 

Staffing Patterns in Urban Education Projects 

The number of elementary and secondary level staff 
who were directly engaged in project activities are shown 
in Table 27. The secondary level staff are divided into 
two categories. ^'Basic Skills^' involve fundamental subjects 
such as reading, writing, mathematics, natural and physical 
science and social science. ''Vocational Skills and Attitudes" 
relate to areas such as business education, home economics, 
industrial arts and other occupationally oriented instruction. 
For each type of staff, both the full-time equivalence and 
the undupUcated number of persons are shown. All direct 
project staff are reported in this item. From the table it 
may be sedn that the largest single number of staff employed 
were paraprofessionals s^t the elementary school level. 



For the completion of the item, the total "full-time equivalent" of staff 
members was calculated. Each staff member was counted as a whole 
of a fraction of the equivalent of an individual who works full time. 
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The number of staff providing supportive services for 
each project was not included in Table 27, ''Supportive 
Services'' staff are those providing auxiliary services to 
the project either to participating pupils or in some other 
way, but who are not directly engaged in project activities. 
There were about 700 individuals who provided these services. 
Adding these to the staff listed in Table 27 we see that in 
order to implement the 218 Urban Education projects, 
approximately 8,800 individuals were employed. It is also 
worth noting that in 187, or 85% of these projects, over 
90% of the teachers employed were certified or licensed. 

Every project activity is expected to have one licensed 
or certified staff member. In addition, some teachers may 
have an aide or an assistant who may provide direct, non- 
professional staff support services. Such additional staff 
are referred to as ''paraprofessionals'\ 

The types of services provided by paraprofes sionala 
may be gleaned from Table 28wliile in Table 29 are shown 
the number of projects using parapr ofessional services. 

Table 28 

Number of Urban Education Projects 
Employing Paraprofessionals 



Mode of Employment 


Number of 
Projects 


Not employed 


43 


Nonins tructional 
classroom duties 


32 


Tutoring 


87 


Small group instruction 


133 


Other 


71 

> 



Responses were received from 2t8 of the projects 
funded and of these the greatest number of projects 
employed paraprofessionals for small group instruction 
while 37.6 percent of the projects used paraprofessionals 
for noninstructional classroom duties. 

Additionally, 43 projects (19.7 percent) did not 
employ paraprofessionals. It may be noted that 61 
projects (28 percent) employed paraprofessionals for 
three or more different services. 



Table 29 

Number of Types of Paraprofessional Services 
Used in Urban Education 



Number of Services 


Number of Projects 


Pe rcent of Total 


None 


43 


19.7 % 


One 


49 


22. 5 


Two 


65 


^9. 8 


Th re e 


' 50 


22. 9 


Four 


1 1 


5. 1 


TOTAL 


218 


100.0 



Component Activity Costs 

The components of all projects were listed. Addition- 
ally, the appropriate major activities were related to the 
respective components. Finally, a dollar amount asso- 
ciated with each component and activity was summed for 
the entire state. From Table 30 it may be seen that the 
single largest component was reading at the elementary 
school level which used diagnostic and remedial activi- 
ties. Table 30 also includes estimates of the average 
cost per participant hour of these activities. Data of these 
types are critical for good decision making in the area of 
internal resource allocation among programs. For example 
the question of at what level of education diagnostic 
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and remedial reading should be offered obviously de- 
pends on at what level such services are most productive 
in terms of student benefits. But in cases where pro- 
ductivity is relatively constant at different grade levels, 
this cost information provides some indication of where 
it can be offered most cost-effectively. 



Table 30 

Component Activity Costs for the Component Activity 
Combination with the Five Highest Expenditures 



Component 


Activity 


Number of 
Projects 


Total 
Cost 


Average Cost 
pe r participant 
hour 


Reading 
Grades 4-6 


Diagnostic 
& Remedial 


32 


$J, 195. 069 


$0. 84 


Reading 
Grades 1-3 


Dlagnos tic 
& Remedial 


23 


2, 300, 646 


0. 83 


Reading 
Grades 7-9 


Diagnostic 
& Remedial 


25 


I, 380, 116 


0. 64 


English as a 
Second Lang. 
Grades 10-12 


De velop- 
mental 


2 


1, 368, 103 


L 78 


Reading 
Grades 7-9 


Develop- 
mental 


9 


I 057, 292 


0, 31 



e. Resource Distribution by Program fypes 

In the school year 1972-73 $227, 575, 72 1 of categorical aid 
was budgeted for use in local school districts. This figure 
includes $46,301,702 for the Urban Education Program. 
The remainder of the funds was allocated under the follow* 
ing other categorical aid programs; E.S.E. A. Titles I, 
II, III, VI-B, the school community Interaction Umbrella, 
and the Experimental Prekindergarten programs. 

Table 31 indicates how local school districts utilized 
Urban Education as compared to the "other" programs 
identified above in terms of project components. The 
data indicate that both the Urban Education and the "other" 
programs emphasized the three priority areas of the De- 
partment - Reading, Mathematics and Bilingual Education* 
The Department expects that the percentages in these 
categories will be even higher in 1973-74 because local 
school districts will have had more time to readjust their 
programs to the priorities. 



ERIC 
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Table 31 

Comparison of Percentages of State Approved Budgets by 
Project Component for Categorically Aided Programs 
in Urban Education Districts 1972-73 



sa:=:--r5SS as - = = = s:.-^ = =r = =s s = = - = = =:=s =s = a=: = s; 

Project Component 


ur oan 
Educat ion 


u c ner 
Sources 


Combined 
Sources 


tveau I ng 


on f\ ftf 
cy ■ U 70 


3/.77( 


36.0% 


Ad ministration 


1 A £. 

10. 0 


4. 3 


5.2 


Matnematlcs 


10. 1 


15.2 


14. 2 


Supportive Services and Others 


10.0 


4.0 


2.8 


Pupil Personnel Services 


8.2 


7.8 


3. A 


English as a Second Language 


7.2 


3.5 


- 

. 5 


Other Basic Skills (Science, Social Studies, 








English, and Foreign Languages) 


A. 8 


9.6 


8.7 


Bilingual Education 


3.7 


6,8 


6.1 


Inscrvlce Education 


3.3 


1.6 


7.9 


Cultural Enrichment 


2.8 


1.5 


•5 


Handicapped 


2.6 


3.6 


1.8 


Evaluation 


2.5 


.0 


5.5 


Adult Career Education 


1.7 


.2 


.3 


Curriculum Development 


1.7 


.0 


.5 


Early Childhood 


.9 


3.2 


1.9 


Vocational Occupational Education 


,8 


.4 


A. 3 


Health Education 


.1 


.6 


.A 


TOTAL 


100. 


100. 


100. 



A large percent budgeted for Urban Education was 
Administration which was nearly Z\ times higher than 
it was (on a percent basis) for all other sources. Re - 
calling, however, that the Urban Education budgets 
represent only 20 percent of all categorical aid budgets 
discussed here and that the problem of purchasing 
administrative services has a minimum cost for a 
project which tends to decrease as a percent, relative 
to a rise in participant enrollment (with a correspond- 
ing rise in direct instructional services), the high 
percent budgeted to administration may be a function 
of , the total budget rather than a trend toward swollen 
administrative staffs. The fourth and fifth largest 
budgeted components are '^Supportive Services and 
Others" {e.g., food> transportation of library aides), 



4 
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and ''Pupil Personnel vServiccs" (e.g. guidance, psycho- 
logical), which combine to 18 percent in Urban FMucation 
as compared to 12 percent on the average for the other 
categorical aid sources. The fifth largest budgeted 
component for Urban Education was "English as a Second 
Language'^ (serving largely Spanish language dominant 
youngsters). Urban Education budgeted over twice the 
percentage amount that the other categorical programs 
budgeted for helping disadvantaged non-English speak- 
ing learners adjust and function quickly in their new 
English language dominant society. 

Urban Education, then, was similar to the other 
categorical programs in that reading and mathematics 
(combined) received the largest portions of the budgeted 
resources. Urban Education budgets did reveal a differ- 
ence in program emphasis from the other programs in 
that a greater percentage was budgeted for pupil per- 
sonnel services, other support services, and English 
as a Second Language, in line with its programmatic 
emphasis. 

In terms of the combined totals for the Urban Edu- 
cation and "other" programs the data indicate that read- 
ing received 36 percen<t of all budgeted monies, mathe- 
matics 14 percent, other basic skills 9 percent, and 
Bilingual Education 6 percent, for a combined total 
of 65 percent. 

Resource Distribution Within Programs 

Table 32 presents a breakdown of the major cate- 
gories of budgeted expense within Urban Education pro- 
grams. Similar information is also provided for other 
categorical aid efforts (enumerated above for purposes 
of comparison)' 

The information displayed in Table 32 reveals that 
the category "Personal Service-Instructional" across 
all categorical programs discussed there involved 
35 percent ($79,624,052) of the total. Fifty-fiv^ percent 
of the Urban Education budgeted funds were allocated 
to the categories "Personal Services -Ins truction", whereas 
34 percent of budgeted funds from Other Sources were 
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allocated to the same categories. The percents of 
budgeted funds allocated to ''Personal Services- 
Noninstructional: and ^'AU Others^' was 15 from 
Urban Education and 44 from Other Sources. VV^ith 
respect to staff costs, these comparisons indicate 
that Urban Education approved budgets were oriented 
more toward supporting instructional costs, while 
noninstructional and other costs were financed to a 
higher extent in the approved budgets of projects 
funded from Other Sources. 



Table 32 



Percent of Std'tn Approved Budgets Ay Budget Category, 
For Cat-gorically Aided Programs in Urban Education Districts, 1972-73 



Budget Categories 



Personal Services-Instruction 
Other Instruction 
Employee Benefits 
Administration 

Personal Services -Non ins tract Ion 
All Others 
Equipment, Supply 
: Evaluation 

Cont, Agree,, Cons. Services 
Trans, of Pupils 
Conf . and Inservice 
Parent Involvement 



Urban 


Other 


Combined 


Education 


Sources 


Sources 


39.2 


33.9 


35.0 


16.0 


.1 


3.3 


10.0 


12.7 


12.1 


8.7 


9.1 


9.0 


8.5 


23.7 


20.7 


6.6 


10.4 


9.6 


6.0 


4.7 


4.9 


2.4 


1.6 


1.8 


1.4 


.9 


1.0 


.7 


.9 


.9 


.4 


2.0 


1.7 


A 


.0 


.0 * 



a/ 

- Includes approved budgets available for evaluation and analysis as of 
October 1 , 1973 



Rounding concealed the $25,920 (less than .1 percent) budgeted for 
parent involvement found in Urban Education budgets. 
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f. Results - The Achieveiyient of Students in 
1972-73 Urban Education Programs 

This section of the report contains a statistical analysis 
of data reported to the Department by school personnel 
for each Urban Education project. 

Departmental staff reviewed the data supplied by 
local districts. Of the 218 projects funded during the 
1972-73 program year, standardized test data from 
114 projects were usable in the v<^tatistical analysis. 
Data for more than one group of participants were 
reported in many projects; in all, there were 469 groups. 
Groups were distinguished for such reasons as multiple 
components within a project, different schools parti- 
cipating in a project, or participants from the four 
grade intervals under investigation. The number of 
sampled participants in the groups ranged from 3 to 
over 2, 000. To assure a greater reliability in the 
pretest and posttest means, groups containing fewer 
than ten participants were eliminated from the statis- 
tical analysis. 

How Progress Was Determined, and by Whom 

Accordi/ig to approved evaluation plans, project 
personnel administered a standardized test prior to the 
beginning of the project treatment (e. g. a reading pro- 
gram) and the same test (usually an alternate form) at 
some specified point during or at the end of the project 
treatment* The first test administration is the pretest, 
and the second is the posttest. For some groups the 
pretest was administered as much as five months prior 
to the beginning of the project treatment* For exarhple, 
a project component conducted in a community district 
of the New York City Public Schools may have specified 
the measvirement of pupil achievement by using reading 
scores obtained from the Metropolitan Achievement 
Test administered during the citywide testing program 



The four intervals studied were early primary (grades 1-3); later 
primary (grades 4-6); early secondary (grades 7-9); and later 
secondary (grades 10-12). Analysis by intervals permits the in- 
clusion of nongraded treatments without extensive data manipulation 
by pupil chronological age/grade placement. 



which OjCCurs in April of each year. If the project 
component did not comn\ence until September, tlien 
there was a five-month lapse between the pretest 
administration and the initiation of the treatment. 
Additionally, if the project lasted for the entire ten 
months of the school year, then only seven months 
of the treatment would be reflected in the posttest 
score. Thus, any difference score, i.e. posttest 
score minus pretest score, is confounded by two 
limitations: (1) the uncertainty of the effect of multiple 
experiences bet>A'een the pretest and the initiation of 
the program, and (2) a diminution of the net gain re- 
ported due to lack of congruence between the testing 
dates and the dates between which instruction took 
place. Large-scale field evaluation commonly suffers 
from limited exercise of the controls specified in 
most experimental designs* Therefore, caution must 
be exercised when deriving conclusions from the data 
and the results of the data analysis. 

Departmental specifications required districts 
to transform participant raw scores to grade -equivalent 
scores and compute the mean grade-equivalent for 
each group on both the pretest and the posttest. District 
staff were required to determine whether the difference 
between the predicted and actual posttest means was 
statistically significant for each group. During the re- 
view of the data. Departmental professionals concluded 
that statistical analyses of the data should be limited 
to tests conducted upon group pretest, predicted post- 
test, and actual posttest means as the most reliable 
sources of data reported by district personnel. 

The Department established a mathematical procedure 
for district staff to employ in predicting each participant's 
pretest grade equivalent score based on the number of months the 
participant had been in school piior to the pretest, and 
the number of months which elapsed between the pretest 
and posttest. District staff calculated and reported the 
pretest mean for each group. Using these means. De- 
partmental staff derived a predicted mean gain and an 
actual mean gain for each group. The predicted mean 
gain is defined as the difference between the predicted 
posttest mean and the actual pretest mean. The actual mean 
gain is defined as the difference between the actual post- 
test mean and the pretest mean. These mean gains were 
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divided by the number of months between the adminis* 
tration of the pretest and the posttest to obtain a month- 
ly mean gain and a monthly actual mean gain for each 
group. These two statistics form the input data for the 
analyses reported in this section and show if the project 
actually could have speeded up students' rate of learning. 

During 1972-73, local education agencies spent up 
to 5 percent of their allocations to evaluate the effective- 
ness of the funded projects. The Department devised a 
plan for assessing the effectiveness of th6 project but 
some districts, especially those in New York City, ex- 
pressed a desire to obtain more information. Some 
districts employed a contractor to undertake the neces- 
sary evaluative work. The types of evaluators selected 
by districts arq exhibited in Table 33. 

Districts outside of New York City {NYC) made very 
little use of evaluation contractors (that is, commercially- 
oriented agencies). Rather, school district personnel 
usually completed the required evaluation work with the 
aid of a consultant. 



Table 33 
Types of Evaluators 





Urban Education 


Type of Evaluator 


Number 


Percent 


Dtstrict Personnel (Outside NYC) 


48 


22.0 % 


NYC Bureau of Educational 
Research (BER) 


6 


2.8 


NYC School Staff 
(Other than BER) 


7 


3.2 


University, flrri, etc, 


157 


72.0 


Total 


218 


100.0 

- 
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Achievenient ProRre ss by Students in the 
Urb an Education Program 

In this section of the report, the record of achievement 
of the Urban Education Program is studied by viewing 
459 subgroups of pupils for which data were available. 
These subgroups were obtained from the total project 
groups by subdivision in accordance with subject studied 
and grade level. Use of the subgroups, or project com- 
ponents, is particularly appropriate because the districts 
reported their data as arithmetic means of the subgroups. 

Pupils were evaluated in two ways: (1) actual gain, 
meaning the gain achieved during the period of partici- 
pation in an Urban Education project; and (2) in terms 
of predicted gain, meaning the gain estimated for the 
same period of time under regular instruction. This 
estimate is based on rate of achievement prior to enter- 
ing the Urban Education Program. The means of these 
two measures were then reported to the Education De- 
partment by the subgroups. 

On the basis of the two measures, it may be said that 
the operation of urban aid has resulted in improvement 
for a subgroup if the actual gain mean exceeds the predicted 
gain mean and as lacking in improvement if the converse 
is true. In addition, the difference--mean actual gain 
minus mean predicted gain--provide9 a measure of im- 
provement. The attached table and charts display the 
data obtained from this point of view, 

In Table 34 are shown total numbers of subgroups 
and associated numbers of these subgroups for which 
improvement has occurred. The succeeding column 
provides corresponding percents. It can be seen that 
there has been improvement for 78 percent of all sub- 
groups, 89 percent of subgroups in grades 4-6 reading, 
and s,o on. 
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Table 34 



« 1 Number and Percent of Project Component Subgroups 
Showing Improvement by Subject Matter 
of Instruction 



s s:ssa&t3sas s a ss as a ts e sss: ss«? ; 



Subject Taught 



Number of ^ Subgroups ^ 



Total Improved 



Percent 
Improved 



Basic Skills 

Reading : 

Grades 2-3 

Grades 4-6 

Grades 7-9 

Grades 10-12 

Other 

Mathematics 

Other Subjects 



40 

53 
112 

78 
9 
12 
94 
61 



27 

44 
100 
58 
9 

12 
65 
41 



68 % 

83 

89 

74 
100 
100 

69 

67 



All Subjects 



459 



356 



78 



NOTE: Improvement - A project is considered as improved if 

actual gain exceeds predicted gain 



In order to obtain a comprehensive measure of parti- 
cipant performance regardless of component treatment, 
a weighted mean was calculated by multiplying each group's 
mean by its number of participants, summing all such re- 
sults^ and dividing by the number of participants. Weighted 
means were calculated for each component and for all com- 
ponents, the results of which appear in Table 35. The over- 
all results showed that the predicted monthly gain from pre- 
test to posttest was 0.61 of a month, i.e. , approximately 
13 days during a 21 -day school month. The corresponding 
actual monthly gain from pretest to posttest was 0. 94, e. , 
approximately 18 days during a 21 -day school month. The 
actual gain surpassed the predicted gain by approximately 
50 percent. Additionally, the actual gain achieved by the 
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Table 3S ' 

Number of Participants and Woighted Hfan 
by I'rojfict Component Cattj^ory 



Project 
Component 


Total 
Numbur of 


Number of 
Sampled 


Weighted Monthly 
Mean Gain 


Part ici pants 


Participants 


Predicted 


Actual 


Mathematics 




3,952 


0,61 


0.90 


Heading 


61,927 


25,p33 
( 

i 


0.56 


0.93 


Other Basic 








Acadt^mic Skills 


10,343 


3,, 224 


0.80 


1.02 


Othtjr Than Basic 








Academic Skills 


27,375 


_ 3^,958 


0.68 


0.94 


All Components 


108,389 


'i6,169 


0.61 


0.94 



sampled participants is approximately that which is ex- 
pected of all school pupils, 21 days in a 2 1 -day school 
month. Because of the nature of the evaluation design, 
which lacked compai?able control groups, the nearly- 
normal gain cannot be attributed explicitly and solely 
to treatments funded under the rban Education Program, 
Many factors contributed to the gain attained by the sampled 
participants, but Miis grcup had at least one common factor 
which was a program treatment funded under the provisions 
of the New York 5>tate Urban Education Program. 

5. T esting to Measure the Effectiveness of Bilingual Education 

The Department is faced with a different set of problems than those 
discussed above when the area of bilingual education testing is examined. 

The Position Paper on Bilingual Education of the Board of Regents of 
the University of the State of New York highlights the education problems of 
an increasing number of public school students whose dominant languages are 
not English. While priorities and strategies for bilingual program develop- 
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mont and class roon^ instruction have been disseminated, and school districts 
and teacher education institutions have a variety of bilingual education activi- 
ties underway, evaluation of the effectiveness of bilingual education programs 
is in a rudimentary stage. 

Recognising this problem, the Department formed a Bilingual Edvication 
Evaluation Committee which functions in concert with a Statewide Advisory 
Committee on Bilingual Education. Participating in the evaluation committee 
are representati vv^s of the Puerto Rican Educators Association, the Office of 
Bilingual Education - Board of Education, New York City, several higher ed~ 
iication institutions involved in bilingual teacher education and research, and 
local district program directors, teachero and psychologists participating in 
operating bilingual education programs. 

The Bilingual Evaluation Committee fulfilled its initial mission by identi- 
fying the following measurement needs in order of priority: 

a. Development of instrumentation which could serve to 
measure the effect of specified bilingual programs on 
student achievement. The recommendation pointed out 
that the greatest need is to measure the degree to which 
linguistic skills taught to students, whose major or only 
language is Spanish, are being successful in equipping 
the student to read, write, understand and speak in 
English. 

b. Development of valid tests which could measure student 
achievement in content areas such as mathematics, social 
studies^ science, etc, when instruction is given in Spanish. 

In addition to these priorities a third suggested priority of the State 

Education Department was to evaluate a sample of bilingual and English as a 

Second Language programs which had a promise of program success in advance of 
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thu instrumentation project (a above), which itself would not have any products 

for field testing until school year 1974-75. 

The following is a report on the progress made in three projects designed 

to meet the previously stated three priorities. These projects are functioning 

as components of the state evaluation system operations in 1973-74 and the 

following descriptions account for their first four months' operations from 

September through December 3i, 1973. 

a. Spanish-English Linguistic Skills Test Development 

The first activity of this project involved researching what 
were the greatest needs in the area of Spanish-English 
linguistics measurement. A determination was made that 
instrumentatiqn was most needed to measure the degree to 
which students i) could comprehend written and spoken 
language in both Spanish and English; 2) could write in both 
languages; and, 3) could speak English and Spanish. The 
second problem identified for study was whether or not 
existing instrumentation developed primarily for Mexican- 
American bilingual students was linguistically transferable 
or adaptable for use by students whose Spanish background 
reflected the Caribbean area, 

After these initial specific recommendations were 
formulated, review and development work was designed. 
Up to the time of this report, a collection of teacher -made 
and standardized and non-standardized tests developed for 
linguistic measurement was made and reviewed, and a 
series of conferences held with teachers and project 
directors of bilingual educational projects to determine 
the degree of instructional emphasis on each of the four 
linguistic areas pointed out above. Based upon a summary 
of these preliminary investigations^ it will be determined 
whether new instrumentation developed or reviewed is needed 
by operating projects for measurement in all the linguistic 
areas mentioned above. 

The most widely used tests in bilingual programs operating 
in the State are the Inter-American series in both Spanish 
and English, the Metropolitan Achievement Test and the 
Wide Range Achievement Teat. Other tests reviewed include 
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those developed by Burt, Spolsky-Murphy , Cervenka, and 
Pimsleur, all of which are in the experimental stage. It 
is planned that certain tests used in the schools of Puerto 
Rico as well as additional teacher-made tests will also be 
examined. This aspect of the bilingual testing operation is 
proceeding as planned and on schedule. 

A continuation of this and the following test administra- 
tion project in 1974-75 calls for State adm" nistration and 
coordination of testing in categorically aided Bilingual and 
English as a Second l^anguage projects whose objectives 
will be measurable by the instrumentation developed. Acti- 
vities for 1975-76 will include the development of alternate 
forms of the final tests , /elected, the refinement of test 
items > and the development of student achievement pro- 
ficiency criteria. 

State Bilingual Testing 

As mentioned previously, a second interim effort is being 
undertaken to evaluate bilingual education projects with 
existing evaluation procedures in advance of the develop- 
ment of new measurement devices. 

As an initial step, all Urban Education funded programs 
designated as either bilingual programs with instruction 
both in Spanish and English, and '^English as a Second 
Language" program's with instruction only in English skills 
for foreign language dominant students were reviewed in 
detail by the Department. Eight projects were identified 
as operating in eight local districts whose proposals had 
evidence of sound instructional practices and whose time- 
table of implementation appeared realistic. An additional 
criterion of selection was that the best possible evaluation 
design should be included as part of the project. The 
districts were New York City Community Districts 23, 24, 
20, 10, 13, 11; the City District of Long Beach on Long Island, 
and the Westchester County City District of Yonkers. 

In these eight projects field visits were made to insure 
that the evaluation design and instructional program would 
be as strong and sophisticated as possible. Arrangements 
have been made to acquire all pre and post test data adjudged 
to be the best available to measure the types of objectives 
and skills taught in each of the projects for State evaluation 
purposes. In many cases, recommendations have been made 
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for the substitution of new teste and instruments for those 
originally planned, in order to strengthen evaluation com- 
ponents, 

A report on this activity will be reviewed early this 
spring. 

Testing of Science Skills when Instruction has 
Been in Spanish 

Although it was determined that the greatest priority need 
in terms of programs was in the development of an instrument 
in the Spanish language which would test those social studies 
and citizenship education skills which had been taught in 
Spanish, the problems of measuring attainment of those 
skills and attitudes as specified objectives in the New York 
State courses of study for social studies indicated that 
initial attempts at measurement development should be in 
another subject matter field. Science education was selected 
because it appeared that reasonable progress could be made 
in developing a new test. Grades 1 through 3 have been 
identified as those which operated more bilingual education 
project components with science instruction in a foreign 
language. The area of Spanish was selected, since it was 
the language of bilingual instruction for the vast majority 
of students in such programs. 

After deciding on the area (i.e. science) an analysis of 
the State syllabus in science for grades 1 through 3 was com- 
nrienced. In addition, a classification system by the content 
of each specified topic of the required study units was begun, 
A further classification of the learning outcomes expected 
of the students, as stated in the syllabus for each unit, was 
followed by a plan for test item writing. Recommendations 
from these activities were then reviewed and plans finalized 
for developing suitable tests. 

In the coming months teachers in bilingual programs 
who are teaching science in the Spanish language in grades i 
through 3 in New York State will be contacted and asked to 
submit single copies of teacher-made tests and any other 
tests available which would measure the various content 
topics and skills as determined in the syllabus unit classi- 
fication system. The work plan calls for the validation 
and refinement of teacher-made test items and the supple- 
menting of any content topics in the syllabus in which good 
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teacher test items cannot be located, by production of new 
test items by project consultants. The product is to be a 
series of question test items which correspond to al4 of 
the topics in the State science syllabus, grades 1 through 
3, and a report of tlie results of a tryout of these science 
test items by a panel of teachers in Spanish science in- 
struction scheduled for 1974-75. 

As of the date of this report 118 teachers in New York 
State have been identified as being involved in such programs 
and as having a high degree of participation and interest* 
It is now anticipated that science tests can be assembled 
by classroom teachers, administered, and results reported 
back during the program year 1974-75, Tentative Depart- ' 
xnent plans in 1974-75 call for the initiating of a project 
to develop similar test items in social studies for the same 
grades and according to the same methodology described 
above in the 1974-75 year, witli field testing slated for 
1975-76, The products produced will be adapted to any 
claanges made in the elementary science course of study, 

B. QUESTION nZ \ WHAT TYPES OF INSTRUCTIONAL PROGRAMS 

(OR PROJECTS EN THE CASE OF CATEGORICAL AID PROGRAMS) 
ARE MOST EFFECTIVE FOR STUDENTS OF DIFFERENT CHARACTER- 
ISTICS? 

In Chapter III - the projected program - tlie point is made that the ab- 
sence of adequate computer software capability has made it virtually impossible 
for most educators to trace what really happens in instruction. In that Chapter, 
the description of the Instructional Evaluation System (lES) indicated how this 
barrier can be eliminated in the future. In the interim program described in 
this Chapter the Department indicates how it is collecting less aggregated in- 
formation tlian has been possible in i-he past but it must be noted that analyses 
described here are not as sound as those that will be possible when the In- 
structional Evaluation System is implemented in all districts. Two additional 



points should be made: first, whenever aggregate information is used, for 
example information describing an instructional project funded under the 
Staters Urban Education Program, one is apt to find as much variation in 
what actually is implemented between classrooms and buildings as is found 
betvveen projects; and, second, there are some professionals who question 
some of the statistical approaches used. On this latter point, the Depart- 
ment will undertake a complete review of these approaches this spring. 

The description of the following five work efforts deals with question #2 
above: 1) New York City Prototype Study of Urban Education and E.S.E. A. 
- Title I Programs - School Year 1973-74; 2) EvaluaHon of Urban Education 
Projects - School Year 1972-73; 3) Urban Education Performance Indicators 
Study - School Year 1972-73; 4) Characteristics of High Performing Upstate 
Schools; and 5) Study of the Effect of Pupil Mobility Upon Pupil Evaluation 
Program (PEP) Results. 

1. New York City Prototype Study of Urban Education and 
Elementary and Secondary Education (E. S. E. A, ) ~ 
Title I Programs - SchooTYear 1973-74 

a. Purposes of the Study 

The purpose of this prototype study is to test the feasibility 
of developing and implementing a Department {as opposed 
to district) managed evaluation program which would also 
be designed to further the development of the projected 
program described in Chapter HI. New York City community 
districts were chosen for the study because of the large 
percentage of funds which they receive from compensatory 
aid programs .(i. e. Urban Education and E. S. E. A. - Title I). 

It was determined that present local evaluation efforts 
although useful for local purposes prohibit meaningful com- 
parison across districts in that each district is free to use 
whatever test(s) it desires, and administer tests at differ- 
ent times of the year. In addition, little is done at the local 
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level in examining the relationship between students receiving 
services under categorical aid programs and those not re- 
ceiving those services. This etudy deals with these two 
barriers and definitely is yielding more comparable data 
available for decision making. The evidence presented on 
pages 4B-60 shows, based on the community district 
sample of all grade five students in reading that questions 
regarding unmet need (i.e. the number of students needing 
services as evidenced by pretest scores list not receiving 
services from categorical aid funds) is now defined. Several 
questions regarding whether funds are allocated to students 
with the greatest need can now be addressed with data. 

Procedures Used in the Study 

A s Jd;jndardized test of reading comprehension (the California 
Test Bureau - McGraw-Hill Comprehensive Test of Basic 
Skills Form Q - Level I) was administered as a pre-test to 
all grade five students in 10 New York City community 
districts in early November of 1973. In May of this year, 
a post-test will be administered. In addition, an experi- 
mental version of the "reading effectiveness measure" will 
also be administered in the spring of this year. The admin- 
istration of this test will assist the Department in refining 
its testing procedures for the "projected" measure. It will 
determine whether by holding the difficulty of test items 
consistent for a passage with a certain "readability", student 
responses follow a set pattern tjo it will be possible to more 
precisely indicate his or her ability to comprehend different 
passages with varying "readability" scores. The scaling of 
the passages for "readability" is not as precise as that de- 
sired in the ultimate measure but is an improvement over what 
has been feasible in the past. 

In addition to these measures of output, an improved set 
of information regarding each project has been collected. 
Specifically, a program questionnaire was completed by 
each Urban Education and E.S.E.A. - Title I coordinator 
in each of the ten community districts for grade five reading 
projects. Twenty-eight questions were used to examine the 
following variables which are thought to have an impact on 
outcomes of projects: i) how well the project objectives 
and instructional procedures are articulated; 2) how much 
of a change in instructional and organizational procedure 
does the project introduce ; 3) how individualized is the 
project; 4) how time intensive is the project (i.e. how many 
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hours are the children exposed to instruction); 5) how labor 
intensive is the project; 6) how well managed is the project; 
and 7) how well is the project documented. The classroom 
teachers involved with the project were asked for additional 
information. vSpecifically they were asked to provide infor- 
mation on the reading textbooks by reading group, the social 
study textbooks used in grade five, an indication of the number 
of students in each ethnic group, and some general information 
/ regarding their experience. Additional information on the 

teachers is available from the BEDS file. It should be noted 
that the purpose for collecting all of this information is for 
analysis of projects, not individual students or teachers. 
The State is only interested in determining to the best of its 
ability what are exemplary projects and in attempting to 
identify the characteristics of projects which are worth 
replicating* 

The information collected in this study is being stored 
on computerized student and program files. This is being 
done to make it practical to undertake analyses after the 
post-testing is completed. A description of the types of 
analyses to be performed which address question #2 follows 
a brief discussion of what the Department is providing 
districts in this study. 

Feedback of Information to Participating Districts 

In undertaking this prototype study, the Department de - 
termined that it should feed back as much information to the 
participating community districts as possible. Generally 
evaluative and descriptive information is provided to State 
and federal agencies but little information is provided for 
the school districts. As an example of how the Department 
is attempting to reverse this practice, after the pretest 
scores were computed by the Department and entered on 
the computerized student record by teacher, printouts such 
as Tables 36 and 37 were distributed to each classroom for 
the teacher's use. In addition, similar tables were provided 
^ to principals and the superintendent. After the post-testing 
similar tables will also be forwarded to the same individuals. 

The participating community districts are also very 
interested in receiving back information from the "experi- 
mental version of the reading effectiveness measure". As 
indicated previously, each classroom teacher indicated what 
reading and social studies textbooks were being used. The 
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"readabilUy'' of Ihese textbooks are now being calculated. 
Their "readability" level will be compared with the stand- 
ardized pretest scorep of students to determine whether 
or not the grade-level difficulties of the textbooks correspond 
to the grade level ability of the students. In addition, after 
the "expe rimental version of the reading effectiveness" 
measure is administered, the Department believes that it 
will be possible to determine whether or not students have 
met the reading standards set by their reading and social 
studies texts. Information from this work will be very helpful 
to districts because generally they individually have not 
previously undertaken this type of effort. 

Planned Types of Analyses - Spring and Summer 1974 

As mentioned previously, in the spring of 1974 (May and 
June), two posttests are planned--!) a standardized post- 
test in reading comprehension will be given; and, 2) an 
experimental version of an effectiveness measure for re d- 
ing will be administered to determine if students are able 
to comprehend English text at a level of difficulty corre- 
sponding to the expectations implied by the assignment of 
particular reading materials. Growth in reading compre- 
hension from November, 1973 to June, 1974 will be evalu- 
ated relative to the actuarial growth in the comprehension 
skills of students in various percentile bands measured 
from data obtained from a national normlng Sample. 

Many other analyses of the data can be executed. These 
potential analyses are of three general types. Each type is 
discussed in subsequent paragraphs. 

Since students in compensatory programs are not ord« 
inarily distinguished from students who are in regular pro- 
grams when standardized tests are normed, statistics such 
as grade equivalents, percentiles, etc. already incorporate 
the gains (if any) prodviced by categorically-funded compen- 
satory programs. Thus, for example, the differences among 
students revealed by standardized test statistics may be 
smaller than they would be if disadvantaged students were 
not participants in categorically-funded programs at the 
time a test was normed. This confounding effect would serve to 
reduce the likelihood of detecting any program effects with 
the tests. In addition, for numerous other reasons, there 
are major methodological problems associated with using 
a national sample (or an urban sample) as a comparative 
basis for evaluating the benefits of Urban Aid programs. 
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However, the problems associated with evaluating 
program effects in relationship to the distribution of test 
scores obtained from a nation? 1 sample can be avoided to 
an appreciable extent if a program versus no-program 
(experimental versus control) design could be employ d. 
In this type of analysis^ those students at each level(per- 
centile band) of pretest achievement who are taking the 
program could be compared with matched students who 
are not in the program. 

The problem is to determine whether designs of this 
type are feasible in the 1973-74 evaluation. That is, it 
will be necessary to review the pre-test data and the pro- 
gram documentation that have been provided by the pilot 
districts in November and December to determine whether 
there are enough comparable students taking and not taking 
compensatory programs. It is the Department's minimum 
expectation that it may be possible to draw subsamples of 
students matched on pretest scores and other variables 
(for example, use of volunteers, all of comparable 
age, sex, and equal proportions of comparable ethnic 
backgrounds) from the available student populations (i.e. 
those in and out of programs). Because of the desirability 
of this type of analysis, the Department is now reviewing 
the pretest results, information concerning who is in which 
program, how students were selected, and the program 
documentation itself in order that a decision to use program 
by-no-program (i.e., experimental versus control) designs 
can be made rationally. 

Another type of analysis that is possible involves the 
comparison of schools within districts or the comparison 
of programs within districts (or both). Such analyses would 
be valuable to determine wjiether one program was more 
effective than another. In addition, on the hypothesis that 
one program proved to be more effective than another, one 
might wish to determine, for example, whether the differ- 
ential application of resources accounted for the result. 
However, such analyses depend upon determining whether 
the programs are run under comparable conditions. There- 
fore, before a decision is niade to undertake analyses of 
this type, their feasibility must be determined by examining 
the ethnic distributions of classes among schools, the pro- 
gram documentation, pretest scores, and so forth. 



-99- 



A third type of analysis is also possible. This type 
involves comparisons between districts based on such 
.variables as staff-student turnover, the quality of 
programs as determined by staff analysis, etc. . Again, 
a large amount of information is being reviewed in order 
to determine whether such analyses are feasible before 
a decision can be made concerning their execution. 

In summary, the pilot approach to Department- 
managed evaluation is as follows. Give a standardized 
pre- and posttest in reading comprehension and a post- 
test measure of effectiveness in reading. Obtain in- 
formation concerning pupil- program documentation. 
Execute the following analyses: 1) a calculation of the 
actual growth of students in each program and an evalu- 
ation of the results relative to the growth expected on 
the basis of actuarial data obtained from a national 
norming sample; and Z) a calculation of the extent to 
which students are able to comprehend text equivalent 
in difficulty to that which they are using in instruction. 
Finally, display all data (e.g., frequencies, means, 
variances, etc.) for reporting purposes, and review 
the results in order to determine the feasibility of: 
program versus no program designs; within-dis trict 
program comparisons; and the feasibility of using both 
BEDS-derived indicators and the results of a program 
standards evaluation for making between-district 
comparisons. (It is apparent that analyses of pre- and 
post-test data can only be undertaken for students who 
took the pre- and posttests. Thus, all evaluation 
studies will be based only upon aiialyses of test scores 
of students who were in the program long enough to 
profit from it. ) 

* Obviously, there is a wealth of information available 

for analyses. The decisions regarding which specific 
analyses are possible and are of a higher priority will 
be made shortly. 

2. Evaluation of Urban Education Projects - School Year 1972-73 

On pages 60-86 , the rationale for the Urban Education Program, 
descriptive information and results associated with the expenditures were re- 
ported. In the following pages, the specific ''exemplary projects^' are summarized 
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since tliey are the projects which have been proven to be successful. The 
compendium contains data on project goals, activities and accomplishments 
and is intended to serve as a guide to local personnel and to answer question HZ 
- What types of projects are most effective for students of different character- 
istics? 

Selected abstracts of projects are presented which meet the following 
criteria: To qualify, an exemplary project is one which equalled or exceeded 
one month gain for each month of instruction, contained valid supporting data, 
had reasonable per pupil expenditures and were funded in some of the priority 
areas* 



a. Reading Laboratory Program 

The project was operated in New York City Community 
District 2 with a budget of $85, 910 and 307 participants 
in grades 4 through 7 (cost/participant $280), 

Major Goal ; To increase reading skills of elementary 
and junior high school students who utilize a reading 
laboratory as measured by the Metropolitan Achievement 
Reading Test. 

Activities ; Reading laboratories were established in one 
elementary and two junior high schools to provide an indi- 
vidualized self -directing, and self -cor recting approach to 
the elimination of reading difficulties. Students worked in 
the laboratory in groups of fifteen with each group scheduled 
for two periods of work each week. Each student was 
diagnosed through testing, and recommendations were form- 
ulated based on the student's needs as evidenced in the 
diagnosis. A schedule of work was prescribed for each 
student. The program made extensive use of innovative 
and programmed materials such as workbooks, language 
arts games and reading skill tapes and employed filmstrips, 
teaching supplies, spoken arts cassette library, controlled 
readers, tape recorders, listening stations and earphones. 
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Findings ! Student reading gains were predicted to be at six 
months but gains were actually achieved at a rate of one 
year and one month. At the fifth grade level the rate of 
growth reached one year and nine months while at the 
sixth grade the rate of growth reached one year and six 
months* These gains were all significant at the , 01 level 
of confidence. For the program as a whole, 65 percent 
of the pupils improved their rate of growth by 50 percent 
or more, 

b. Operation Reading Success 

The project was operated in New York City Community 
District 19 with a budget of $748, 021 and 1, 502 participants 
in grades 4 through 6 (cost/participant $499). 

Major Goal ; To diagnose and correct reading deficiencies 
to achieve a growth of one year in one year. 

Activities ; Each school participating in Operation Reading 
Success was equipped with a reading laboratory staffed by 
a teacher and two educational assistants. Each laboratory 
was designed to diagnose and correct specific reading de- 
ficiencies in a small group setting. Participants attended 
the laboratory for one hour a day, A class of 16 pupils, 
grouped by ability, was in attendance at any one time. 
There were four such classes in each of 19 schools. The 
laboratory included three stations composed of a listening 
station, an Aud-x station, and a combination of a controlled 
reader and TACH-X stations. Each station was designed 
to correct specific reading deficiencies and improve, 
stimulate and motivate the participating child. 

Findings ; The mean pretest score for the group was 3.09 
and the posttest score was 4.01. Sixth grade students 
achieved at a rate of one year and one month. The average 
rate of growth historically was four months while the actual 
growth was significantly higher at the .001 level. 

c. Junior High Reading Laboratories 

The project was operated in New York City Community 
District 22 with a budget of $161, 056 and 473 participants 
in grades 6 through 9 (cost/participant $340). 
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Major Goal : To improve reading ability of pupils who are 
one or more years behind so that they are able to return 
to rej^ular classes. 

Activities ; Each laboratory was supervised by a reading 
teacher who was aided by two educational assistants. The 
students were programmed for a half year with five double 
periods or five single periods a week. Those students who 
did not achieve a singificant gain in a half year remained 
for a full year. In September each pupil was given a pretest 
with the Metropolitan Achievement Test in Reading. A diag- 
nosis was made of each student's needs using the Gilmore 
Oral Reading Test and the Roswell Chall Phonic Test. 
Every student^s eyes were tested with the Titmus Vision 
Tester. A program was planned individually for each student 
based on his deficiencies. Students worked individually or 
in small groups. The teacher and assistants taught and gave 
help as necessary. After testing, students returned to their 
English classes if recommended by the teacher. 

Findings ; Ninth grade students attained one year and one 
month's growth, eighth grade students attained one year 
and four months* growth while seventh grade students 
attained nine months' growth. } redicted growth was four 
months and all groups exceeded this rate at the .001 level 
of confidence. 

Diagnostic Reading Program 

The program was operated in New York City Community 
District 27 with a budget of $559, 972 and 1, 589 partici- 
pants in grades 2 through 9 (cost/participant $352). 

Major Goal ; To improve the reading level of students in 
grades 2 through 9 after diagnosis and specific skill in- 
struction. 

Activities ; Second grade children who scored below grade 
1. 5 on the Metropolitan Achievement Test were grouped 
according to similar needs for small group instruction. 
They received instruction in listening skills^ auditory and 
visual discrimination and practice in decoding and compre- 
hension skills utilizing th ^ Flashcard Reader, cassette 
player and listening centers. In the Reading Helper aspect 
of the program upper level children reading at least six 
months below grade level as measured by the Stanford 
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Diagnostic Reading Test were given supplementary instruc- 
tion in reading skills by paraprofessionals. The Reading 
Helpers worked with a maximum of three children at a time 
who were given specific help. Each child was screened 
for visual deficiencies using the Keystone Telebinocular^ 

Findings ; Achievement ^ains for the eight group 
averaged one year as measured by the Metropolitan Achieve- 
ment Test, Achievement increase at the fifth grade was 
one year and four months and at the sixth grade it was one 
year and seven months, both significant at the , 01 level 
of confidence. 

Cluster Team Program 

The project was operated in New York City Community 
District 31 with a budget of $157, 008 and 500 participants 
in grades 6 through 8 (cost/participant $3i4). 

Activities ; The program was designed to provide a learning 
environment and instructional methods for students with 
serious reading handicaps. The children in the program 
were economically as well as educationally deprived. 
Pupils in grades 6 through 8 met at least twice a week with 
the reading teacher* A teacher assistant provided small 
group instruction. The instructional method employed 
multi-media and multi-sensory equipment and techniques. 
The student first received perceptual accuracy and visual 
efficiency training followed by activities which were intended 
to enrich his experiential background and prepare him for 
subsequent instruction. The student then developed vocabu- 
lary, word recognition, word attack skills^ and comprehen- 
sion skills. Finally, all of the words, skills, and concepts 
taught during the skill-building sequence were applied by 
the student during independent reading* Achievement was 
measured by use of the Metropolitan Achievement Test. 

Findings : The average achievement increase for grades 6-8 
was one year and three months* 

Home Study Center for Disadvantaged Children 

The project was operated in Binghamton City Schools with 
a budget of $20, 190 and 393 participants in grades 1 through 
12 (cost/par ticipant $5 1 ). 
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iVta jor Goals ; To provide tutorial assistance for needy 
scudenls, to diagnose special problems, lo provide 
individual help and with parents to encourage students to 
do better in school. 

Activities ; A minimum of one hour of individual tutoring 
per week was provided in the home where possible and 
when not possible at a Homework Center. Tutors were 
either local high school students, college students or 
volunteer adults from the community. The homework 
centers were equipped with appropriate materials for 
study and an atmosphere conducive to study. Liaison 
was established with school officials and faculty for 
purpose of mutual cooperation and resolution of academic 
problems of tutees. Children in grades 1 through 12 were 
served. The staff -participant ratio in project activities 
was 1 to L 

Findings ; The improvement in reading and mathematics 
was one year and one month which was significantly greatei; 
than the historical rate of growth for the students. 

Prescribed Reading Essentials Program 

The project was operated in Ossining Public Schools with a 
budget of $19, 727 and 150 students in grades 7 through 9 
(cost/participant $131). 

Major Goals ; To improve reading vocabulary and compre- 
hension as measured by the California Reading Test after 
a program stressing individualized instruction. 

Activities ; Teachers conducted reading classes ranging 
in size from 1 to 10 students* Classes were occasionally 
conducted for the groups but usually students worked in- 
dividually after having been appropriately programmed 
according to their special needs and levels. The teachers 
conferred with parents of participants and a building- 
parent group was involved in the process. The teacher 
aide supervised students working on individual tasks and 
occasionally helped the teacher organize classroom 
materials. The target group was ninth graders reading 
below level after which other students were invited to 
participate. Students were scheduled for reading two or 
three times a week. 
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FindingS! The average achievement gain was one year 
and tNvo months. 

Uplifting of Basic Skills 

The project was operated in the City School District of 
Troy with a budget of $90, 000 and 500 participants in 
grades 1 through 6 (cost/participant $180). 

Major Goal ; To improve skills in reading, mathematics, 
science and social studies through a program which 
stresses reading skills, increased interest in reading, 
resource materials which children can use for independent 
study and teachers can use in their classrooms and by 
providing a resource teacher to aid and motivate children 
in independent study. 

A cttvities : Various instructional methodologies and pro- 
gram materials were applied across the group. In one 
case, forty-five children in grades four through six were 
helped by a remedial reading teacher. Grouping in this 
instance was based on individual weaknesses rather than 
by grade with emphasis placed on comprehension, phonics, 
word building and vocabulary. In another part of the pro- 
gram, thirty children who had not been achieving up to 
their capacity worked in a large room which could be 
divided for small group instruction. Team teaching and 
individualized instruction were used for teaching reading 
and mathematics with emphasis on work skills, vocabulary 
and computational skills. The overall district program also 
included a library-instructional media resource center which 
provided materials for remedial reading components. 

Findings: A sample of 141 students was tested using the 
Metropolitan Achievement Tests for grades 4-6. Achieve- 
ment gains were found to be one year and two months and 
results were statistically significant at the ,02 level of 
confidence. 

Secondary Education-Reading Programs 

The project was operated in the Utica Public Schools with 
a budget of $165, 568 and 965 participants in grades 7 through 
12 (cost/participant $176) 



Major Goal ; To develop reading skills and promote 
positive attitudes toward reading. 
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Activities; At the high school level a reading center was 
operated in laboratory format with a reading teacher and 
assistant. The reading teacher was responsible for 
diagnosing reading skill needs and for planning, develop- 
ing and implementing instructional activities to improve 
those skills. The" assistant helped students in the use of 
equipment and materials, provided tutorial help and 
maintained records and materials. The effectiveness of 
the center in the improvenient of reading was measured 
with the Stanford Diagnostic Reading Test and the Stanford 
Achievement Test* 

In the junior high school two types of reading centers 
were used. One type consisted of an after-school program 
utilizing student tutors. Students were scheduled for three 
one-hour sessions per week for 15 weeks. The aim of 
each session was to improve reading skills through 
positive tutor student interaction and varied learner 
activities utilizing multi -media and multi-level approaches. 
The other type center was a part of the regular day 
program which emphasized group instruction. 

Findings; Average achievement gain at the ninth grade 
level was one year and five mon \s. This is statistically 
significant at the . 01 level of confidence. 

Community School 

The project was operated in White Plains Public Schools 
with a budget of $43, 241 and 112 participants in grades 8 
through 12 (cost/participant $386) 

Maj.or_Qoa|j To carry out an alternative educational program 
at the secondary level for students not reached by existing 
secondary school programs. 

Activities; Students participated in academic work three 
days each week and small group career orientation for 
the other two days.^ The staff consisted of a director, a 
community liaison coordinator, four full-time and five 
part-time academic teachers, a guidance counselor, a 
part-time social worker, and fifty -two volunteer teachers 
who provided community courses. The school offered a 
full range of academic offerings as well as field experience 
courses directed toward career and self-exploration. Close 
personal contact was provided between teacher and students 
in math and reading instruction. Students were given 
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prompt assistance with their work, as well as frequent 
consultations with teachers in the diagnostic and remedial 
phases of the program. Readily available manipulative 
aids were provided in mathematics. 

Findirij ^s; Students achieved at a rate of 1.4 months for 
each month of participation in the program. 

In addition to Urban Education funds, all Urban 
Education Districts receive ESEA Title I funds to pro- 
vide assistance for educationally disadvantaged pupils^ 
Some ESEA Title 1 projects in these Urban Education 
districts were selected as exemplary. The criteria for 
selection of <hese projects was the same as those 
previously described for Urban Education. 

Bilingual Piogram 

The project was operated in New York City Community 
District 3 with a budget of $145, 857 and 425 participants 
in grades K through 3 (cost/participant $343). 

Major Goal ; To demonstrate a significantly different mean 
on the posttest measure of Spanish reading than on the 
pretest. 

Activities ; Eighteen bilingual classes were organized in 
a language arts curriculum. Reading content included a 
portion of cultural heritage material, an action that had 
strong endorsement by parents. The project operated for 
40 weeks at hours per week* Each participant received 
520 hours of supplementary instruction. Seventeen teachers 
and 22 paraprofessionals were involved. 

Finding s; The Cutler Diagnostic Spanish Reading Test consists 
of 5 subtests, all of which use raw scores as measures of 
accuracy. For each subtest, the posttest mean was found 
to be significantly different from the pretest. Thus, the 
data led the evaluator to conclude that target population had 
achieved considerable growth in ability to read in Spanish 
between the time of the pretest and posttest. 

Mathematics Laboratories 

The project was operated in New York City Community 
School District 31 with a budget of $223, 846 and 357 par- 
ticipants in grades three, four, five, and seven (cost/ 
participant $627)* 
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Major Goal: To achiovc beyond expectation in math- p'^a^u't 
concepts and computation as measured by the Starforc! 
Diagnostic Arithmetic Test. 

Activities : A diagnostic analysis of pupil deficiencies was 
followed by prescriptive activities planned by the laboratory 
staff and classroom teachers. The physical maaioulation 
of materials was emphasized for concrete exof^r'pnce prior 
to abstractions. The Mathematics Laboratories were 
conducted for 36 weeks for 35 hours per week. Individuals 
participated 5 hours per week for 36 weeks which orovlder^ 
180 manhours of math instruction supplement per pupil. 
The equivalent of 9 full-time teachers and 8 nonprofess vjnals 
were employed in the treatment. 

Findings : The target population averaged 18 months growth 
in achievement in a 7-inonth period between the October 
pretest and May posttest administration of Ihe Stanford 
Diagnostic Test. Growth expected without the ESEA, 
Title I treatment ranged from 3| to 6 months. 

m. Services to Early Elementary Children 

The project was operated in New York City Community 
District 15 with a budget of $1, 403, 789 and ^, 599 par- 
ticipants in grades Kindergarten through 2 (cost/ 
participant $540). 

Major Goal : To achieve a significatn difference in 
reading between pretest and posttest means as measured 
by the Metropolitan Reading Readiness Test. 

Activities: A programmed instruction approach using the 

DISTAR method was used. Three hundred forty reading 

lessons were given. The project operated 36 w^eks for * 

30 hours per week. Each individual received 180 hours 

supplementary treatment. The project employed 46 

teachers and 157 aides. 
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Findings: Random samples {50%) from each grade 
achieved significant differences between pretest and 
posttest means. In second grade, California Achieve- 
ment Test scores indicated a mean of 9 months growth 
in the 7 months between pre and posttest administrations. 



Bilingual Program 

The project was operated in New York City Community 
District 19 (Brooklyn) with a budget of $1, 010, 212 and 
2,600 participants in grades 1 through 8 (cost/ participant 
$389). 

Major Goal ; To achieve a significant irnprovement in 
Spanish reading as measured by the Cutler Diagnostic 
Test in Spanish, 

A ctivities ; Participants received English as a Second 
Language instruction alternately with reading ir.s traction 
in Spanish. The project operated 40 weeks for 32 hours 
a week, employing 59 teachers and 12 aides. Participants 
received 1,040 hours supplementary treatment. 

Findings r A sample of 303 students pretested in January, 1973 
yielded an average raw score of 73. 9 from a possible 112 
points. The post test mean of 97. 1 obtained in June, 1973 
was found to be significantly different at the . 01 level. 

Reading Diagnostic Center 

The project was operated in New York City Community 
District 29 (Queens) with a budget of $312, 605 and 625 
participants in grades 1 through 4 (cost/participant 
$300). 
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Major Goal ; To achieve in reading at a rate ol 1 naonth 
growth for each n^iontli of participation as measured by 
thg Spache Diagnostic Reading Test. 

Activities; A clinical approach with psychological and 
reading diagnosis was followed by prescription of skills 
related to code breaking. The project operated 40 weeks 
for 30 hours a week. The participants received 200 hours 
of supplementary instrviction. The staff included 9 teachers, 
9 educational assistants, 1 psychologist, 2 social workers , 
and 1 family assistant. 

Find ings ; Among the participants for which both pretests 
and posttests were available, 40 participants in grade 1 
achieved 10 months in the / months between tests, 195 
grade 2 participants achieved 14 months; 274 third graders 
achieved 14 months, and 35 grade 4 participants achieved 
13 months. 

English as a Second Language in Nonpublic Schools 

The project was operated by the New York City Central 
Board with a budget of $634, 392 and 3, 976 participants 
in grades 1 through 12 {cost/participant $160). 

Major Goal : To significantly improve English language oral 
fluency by pupils with limited or no English speaking 
competency as measured by the New York City Board of 
Education Pupil Ability Scale. 

Activities ; Direct instruction was provided in listening compre- 
hension and oral facility. Materials included tapes, film- 
strips, phono viewers, as appropriate to age-interest levels. 
The project operated 40 weeks for 5 hours per week. Each 
pupil received 200 hours of supplementary instruction. The 
staff was composed of 42 teachers. 

Findings : The mean rating of a sample (372 participants) 
stratified by grade rose from 5. 836 to 8« 043 on the scale. 
The scale consists of a rating schedule of 1 to 6 points. 

Remedial Reading Clinic 

The project was operated in Buffalo with a budget of 
$147, 338 and 423 participants in grades 3 through 6 
(cost/participant $348). 
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Major Goal ; By the conclusion of the project year, all 
students participating were expected to demonstrate a 
significantly greater rate of growth in reading during 
the 10-month period of the program than they had demon- 
strated in the past on the Reading subtest of the Metro- 
politan Achievement Test. 

Activities ; A clinical team observed and tested pupils* 
Tins provided diagnosis and prescription for a treatment 
carried out through a small group or tutorial setting in 
twelve target area schools. The treatment ran 40 weeks 
for 15 hours per week. Individual participants received 
120 hours of supplementary instruction. The staff included 
one reading specialist, 16 reading teachers, 16 school aides, 
4 reading diagnosticians, and 4 psychologists. 

Findings ; A sample of 423 participants who were expected 
to achieve 4| months' growth in ten months actually achieved 
14 months^ growth in this period as measured by the Metro- 
politan Achievement Test (reading). 

Corrective Mathematics 

The project was operated in Buffalo with a budget of $541,821 
and 2, 058 participants in grades 2 through 8 (cost/participant 
$263). 

Major Goal : To demonstrate a significantly greater rate of 
growth during the lO-month period of the program than had 
been demonstrated by students previously measured by the 
Metropolitan Achievement Test, 

Activities : A tutorial approach and a laboratory approach 
were employed with different participants. The most severely 
disadvantaged pupils participated in the lab sessions, while 
participants in the 2nd and 3rd stanines received tutoring. 
The treatment lasted 37 weeks for 27 j hours per week with 
the learner receiving 111 hours supplementary math instruction. 
Thirty-seven math teachers, 27 teacher aides, and 27 support 
pupil personnel service staff were employed in the project. 

Findings ; Two hundred forty-two pupils, whc were predicted 
to achieve 5 months' growth, participated in the math labs* 
These pupils actually attained 14 months' growth as measured 
by the combined Metropolitan Achievement Test. The 1,082 
disadvantaged learners in the tutorial sessions, who were 
predicted to attain 5 months without tutoring, achieved 15 
learning months as a result of the program. 



Urban Education Performance Indicators Study - School Year 1972«73 

a. Purpose of the Study 

For several years, the Departnient has been developing the 
Performance Indicators in Education (P. I. E. ) project as 
a statistical method of determining how various character- 
istics of the district relate to achievement. Generally, the 
data used in the statistical model has been BEDS (Basic 
Education Data System) information and PEP (Pupil Evalua- 
tion Program) scores. ^As part of the evaluation for the 
present year, this work is being extended to examine speci- 
fic Urban Education projects* The purpose is to develop 
Urban Education performance indicators to determine how 
various characteristics of projects relate to achievement 
on a project rather than on a.district basis. 

b. Procedures to be Used in the^Study 

Three plans of analyses have been developed to investigate 
the following: 1) the feasibility of studying reading and 
arithmetic achievement by simultaneously developing 
-equations for both criteria; and, 2) the effects of program 
components, activities, and objectives; of program 
duration and of pretest scores on achievement. 

The analyses are based on data on compensatory aid 
projects as submitted to the Department in the Mailed 
Information Reports (a data collection instrument used by 
the Department to collect descriptive and evaluative in- 
formation from local school districts) for the 1972-73 
school year. Selected data for 80 projects have been 
extracted from MIR^s and are being analyzed. Analysis 
of additional data for these 80 projects and of data for 
other compensatory projects is dependent upon the avail- 
ability of data being assembled by the Bureau of Urban 
and Community Programs Evaluation, 

c. Anticipated Results of the Study 

1) Information concerning the effects of program compo- 
nents, activities, and objectives and the interaction 
of these factors. For example, the results might 
show that the objective of improving reading achieve- 
ment has been achieved "Better through activities A^ 
B, and C than through activities X, Y, and Z in a 
population with given characteristics. 



2) Information concerning the effect of program duration 
on student achievement. 

3) Information concerning the relation of pretest scores 
to achievement. 



In the coming years, this type of study will be expanded 
to cover each school year with the explicit purpose of de* 
termining those project characteristics which have different 
effects on students under various conditions in different 
projects. As a result of these analyses, unmeaningful items 
can be deleted from the MIR forms and dissemination of in- 
formation regarding successful projects can be improved. 

Study of the Characteristics of High Performing Upstate Schools 

a. Purpose of the Study * 

The purpose of this component is to identify characteristics 
of upstate schools which are highly successful in promoting 
student achievement given the conditions under which they 
operate. 



b. Procedures to be Used in the Study 



A project is presently being finalized with the Bureau of 
Institutional Development of SUNY at Buffalo. That organi- 
zation already has received the following data through a 
separate arrangement with the Information Center on Edu- 
cation: 1) Basic Educational Data Systems file for 197rfor 
all schools in the State; 2) Pupil Evaluation Program files 
for 1969-1972; 3) Financial File for all districts; and 
4) Personnel File for all districts. The Bureau .7i Institu- 
tional Development is presently merging these files. 

This work complements on the school building level 
activities of the Bureau of School Programs Evaluation to 
develop performance indicators on the school district level. 



Specific objectives of the project are: 

1) To identify schools which are achieving above expected 
levels in elementary school reading and arithmetic. 

2) To identify those school and community characteristics 
which are related to varying school performances in 
reading and arithmetic. 

The Department is using funds to refine an observational 
instrument which v/as used in a study of urban schools, and 
to test it in a number of upstate schools. Proposals have 
been solicited from potential contractors and consultants. 

Anticipated Results of the Study 

The major anticipated outcomes are: 

1) Information on which BEDS data items which can account 
for a degree of school performance. 

2) A list of school management and instructional practices 
to be examined further for their relation to student 
achievement* 

3) An observational instrument designed to examine 
management and instructional activities in nonurban 
schools, 

4) A plan for a study in depth of student, community, and 
program factors related to school performance. 

This study will be expanded in fiscal year 1974-75 to 
identify school management and instructional practices 
related to school performance in upstate schools. This 
rationale for this work effort is to verify judgments on 
activities and conditions which promote student achievement 
since they are presently confounded by the many interrela- 
tionships between school factors, nonschool fa<:tors, and 
student characteristics* The Performance Indicators 
project is an effort to separate these factors so that those 
which can be affected by school personnel, primarily the 
school program factors, can be examined to determine how 
achievement can best be enhanced. The study underway and 
proposed for 1974-75 carries this process further by examining 
a number of factors at the school, rather than on the district 
level. 
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This stream of inquiry is designed to make statewide 
Performance Indicators more useful as a school management 
tool. In 1975-76> this project would l)e phased out as a 
separate endeavor and its findings integrated into the Urban 
Education Performance Indicators stream, as described 
above. 

Study of t he Effect of Pupil Mobility Upon 
Pupil Evaluati o n Program (PEP) Results 

a. Purposes of the Study 

Each year, after the PEP test results have been reported 
to the schools, it is not uncommon to have principals 
complain that because they have a large pupil turnover, 
they are being unfairly "blamed^' for the educational de- 
ficiencies of pupils who have been in their schools for only 
relatively short periods of time. 

Of course, what these principals fail to realise is that 
it is not at all a matter of '^blame'^ The primary purpose 
of the PEP program is to identify and locate a target group 
of pupils for special educational programs. If pupils are 
properly classified in the target group, on the basis tJiat 
their scores fall below the Statewide Reference Point 
established for that purpose, how long the pupils have been 
enrolled in the school is not a particularly important con- 
sideration for purposes of educational planning. The point 
remains that students in the target group have certain edu - 
cational needs, and the number of target group pupils in a 
school is an index of unmet needs. The validity of this 
index is in no way diminished by the fact that more or less 
of the pupils are transferees. 

Nevertheless, it is true that there has been a tendency 
in some sectors of the press and the public to equate the size 
of the target group in a particular school with the effective- 
ness of the educational program. Certainly that error would 
be compounded if pupil mobility were ignored. It was de- ♦ 
termined, then, that it would be useful to explore the dimen- 
sions of the mobility problem, and the relationship between 
pupil mobility and PEP scores. 

b. Procedures Used in the Study 

To provide a picture of pupil mobility, a sample of public 
schools participating in the regular October 1972 PEP test- 
ing program were requested to provide two separate summaries 
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of reading LesL scoret> lor Grade 3 pupils, as follows: 

1) all pupils enrolled, including transfer students, and 

2) only tlioso stvideiUs who ]\<Xi\ bocMi continuously enrolled 
in that school since beginning Grade 1, 

The sample was drawn on a stratified basis. In 
New York City, the sample consisted of all schools in 
Local Districts 6, 17, and 24, selected because their 
ethnic composition and October 1971 test scores were 
generally typical of New York City as a whole* Of the 
7 large city districts in the State, 6 were included in the 
sample; and of the 8 medium-size city districts, 7 were 
included. In the small-city, village-suburban, and rural 
community types, roughly 10% of the school districts in 
each type were included, again selected on the basis of 
the criterion of representativeness in the October 1971 
testing. Table 38 indicates the number of districts, 
school buildings, and pupils in each community type in 
the sample. 

Table 38: Districts, School Buildings, and Pupils in the Sample 





School 


School 




Community Type 


Districts 


Buildings 


Pupils 


N«\v York City 


3 


43 


7, 004 


Large City 


6 


204 


13, 963 


Other Cities 


13 


127 


7, 916 


Village -Suburban 


25 


80 


8, 031 


Rural 


38 


55 


3, 609 


Total 


85 


509 


40, 523 



c. Results of the Study 

Table 39 indicates that transfer students are a common 
phenomenon in the schools of the State, even as early as 
Grade 3. The highest rate of transfer students was found 
in the sample of New York City schools, where a total 
of 37% of the Grade 3 pupils had enrolled initially in 
Grade 1 in some other school. The lowest rate vyas found 
in the rural schools, but even here the rate of transfer 
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among Grade 3 pupils was 19%. When the various 
community types are weighted for relative enrollments 
in the State, the transfer rate for the State as a whole 
in Grade 3 may be estimated at 29%* 



Table 39j Relative Frequency of Third Grade Transfer Pupils 



% of 






Large 




other 




Villag 


c 






Transfe r 


NYC 




Cities 




Cities 


Suburban 


Rural 


Pupils 


Schools % 


Schools % 


Schools % 


Schools % 


School 


s % 


Over 50% 


2 


5% 


21 


10% 


15 


12% 


2 


2% 


3 


5% 


49-40 


13 


30 


27 


13 ' 


10 


8 


4 




1 


2 


39-30 


17 


40 


49 


24 


21 


16 


12 






2 


29-20 


9 


21 


59 


29 


38 


30 


28 


3? 


lb 




19-10 


1 


2 


37 


18 


38 


30 


27 


34 


?J 




Below 10 


1 


2 


11' 


jb_ 


J_ 


4 


7 


_9_ 


_7_ 


13_ 


• « 

Total 


43 


100% 


204 


100% 


127 


100% 


80 


100% 


55 


100% 



% of Transfer 

Students in ^ ____ 

Combined 

Group 37% 34% 26% 26% 19% 



However, it must be kept in mind that the transfer 
rate varies rather widely among individual schools. Some 
schools in each community type have third grades made 
up of 50% or more of transfer students, wliile other schools 
have third grades made up of fewer than 10% of transfer 
students. 

That pupil mobility is indeed related to pupil achievement 
is suggested by a comparison of the reading test results of 
transfer and non-transfer pupils {Table 40), In New York 
City, for example, whereas 57% of the transfer students in 
Grade 3 had scores below the Statewide Reference Point for 
identifying target group pupils, the corresponding target 
group rate for non-transfer studen^ts was only 36%. Outside 
of New York City, however, the relationship was less marked. 
It would, therefore, appear that the problem of transfer 



studerds varies witli th<j type of schooK Outside of 
t\ew York CJity, foj* example, transfer students may 
I^CJ^erally \)v CAm\\np^ fvc>n\ schools with educational 
pro^;ravns eoniparablc It) that ol tiie new school, where- 
a,^ in Now York Ciiy tVie transfer students may be more 
likely to be educati otwxlly disadvantaged in comparison 
with non- transfer students in the same school. 



Table 40 

Comparative Test Results for Transfer 
and Non- Transfer Stvidents 



Percent Target Group Pupils Below Statew ide Re/ erence Point 





Total 


Non- 




Diffc i-ence: 




Etiroll- 


Transfer 


Transfer 


I'raiisfer Minus 


ComnAuuity Type 


meni 


Pupils 


Pupils 


Non- Transfe r 


New York City 


44% 


36% 


5 7% 


21% 


LargcCl ties 


36 


33 


43 . 


10 


Other Cities 


21 


19 


26 


7 


Village -Subu rl?an 


14 


13 


17 


4 


Rural 


18 


17 


22 


5 


Statewide Estimate 




21% 


36% 


15% 



Granted that the reading test scores of transfer 
students in a school may tend to be lower than the scores 
of non* transfer students, the question still remains 
whether this factor has a significant effect on the com- 
parison of any particular school with other schools in 
the same community type. That is, assuming that all 
transfer students were systematically excluded from 
PEP test reports, it might be expected that the target 
group rates in all schools would tend to be lowered, 
But if the degree of change in target group rate is fairly 
constant, then the inter -school comparisonp within a 
particular type would remain relatively high wlien transfer 
students are eliminated, and schools with unusually low 
target group rates would remain relatively low. 
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To determine what the practical effect would be if 
transfer students were eliminated from PEP test reports, 
the correlation was found for each community type bct\veen 
the target group rate in the total enrollment and the target 
group rate among non-transfer students only. Table 41 
indicates that among schools in large cities, for example, 
the correlation was .96, That means that, with reference 
to the percentage of target group pupils, the relative stand- 
ing of any particular school among all other schools in large 
cities was virtually the same whether transfer students were 
included or excluded. Even witliin the same large city, as 
well, the relative standing remained unchanged. Further, 
very much the same picture was found for New York City 
schools and for schools in all other community types. 



Table 41 

Correlation Between Percent of Target Group Pupils 
in Total Enrollment and Percent of Target Group 
Among Non-Transfer Pupils 



School Correlation 

Community Type Buildings (Pearson r) 

New York City £3 . 93 

N. Y. C. District 6 (11) ^ (. 93) 

N. Y. C. District 17 (13) (. 92) 

N. Y.C. District 24 (19) (.91) 

Large Cities 204 , 96 

Albany (17) (. 95) 

Buffalo (68) (.97) 

Rochester (40) (.91) 

Syracuse (31) (-95) 

Utica (19) (. 98) 

Yonkers (29) (,98) 

Other Cities 127 , 94 

Village-Suburban #80 .95 

Rural 55 .94 
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Suniniary of the Findings 

The frequency of transfer students, defined as students 
who began enrollment in Grade i in another school, is 
fairly high even by the time that the students reacli 
Grade 3. In the State as a whole, it is estimated that 
about 29% of the Grade 3 students tested in the October 1972 
PEP program were transfer students. As a group, the 
transfer students did in fact tend to have lower PEP read- 
in^^ test scores than the non-transfer students. However, 
this student mobility factor did not seem to have a suffi- 
ciently differential effect on the PEP results of schools 
in the same community type to affect the relative standing 
of individual schools. That is, if transfer students had 
been eliminated from the PEP reports, the relative rank 
order of individual schools in Grade 3 reading scores, as 
compared with other schools of the same community type, 
would have remained virtually unchanged. 



IIL DESCRIPTION OF PROGRESS IN DEVELOPING 

THE PROJECTED PROGRAM; A COMPREHENSIVE 
STUDENT EVALUATION PROGRAM 

The Department's '^projected'' program described in this Chapter has 
been developed over the past three years. Many factors were considered. 
First, question3 such as those outlined in the introduction were formulated. 
Second, needs of decision-makers at various levels and interested parties 
such as the Legislature, parents and the general public were examined in the 
context of what information was needed to improve decision-making, and to 
Improve reporting to parents and the public. Third, and most important, 
the answers to all qviestions were pursued with a view to improving the educa- 
tional system so that stxidents would receive a better education- Fourth, re- 
views were made of existing systems Msed in this State and elsewhere, speci- 
fic analytic research and developn\ent activities undertaken, and pilot tests 
wore ittiplemented to test out new approaches thought to be siiitable. In addi- 
tion> two "state of the art" reviews were seen as being crucial to developing 
this "projected" program* The reviews of (1) existing testing methodologies, 
and (2) availability of disaggregated information are presented under "A" below. 

All of the work was conducted within what is technically labeled as a 
"systems approach". The Department quickly realized that evaluation of 
instructional programs could not be isolated from other aspects of the instrvu:- 
tional process such as instructional program planning, instructional manage- 
ment, or the development of an information system able to provide "feedback" 
data on the results of implementing various instructional approaches and 



utilizing various combinations of rt'^ovirccs, Vho ''systems approacli" allowed 
the Departrneiit to view all these* needs in an interrelated manner. 

A. STA' n:: OV THE ART REVIEW' S 

' • ^'^^j ^V^^'^^- Methodoloi;les; Is the Educationa l S ystem S uccecdinj^? 

The Department's review of e>dsling testing methodologies revealed 
that existing testing methodologies are not adequate for addressing tl)e first 
question outlined above-- To what extent are New York S tate's student s succ ess- 
fu lly mastering basic skills (e.g, reading, mathema tics)? It has been gener- 
ally accepted that standardized tests, which are the most widely used form 
of across-district testing in this State and throughout the country, are adecjuate 
measures to answer the question. This is not the case. Scores on such tests 
do not provide infori^iation concerning the outcomes of insitruction because 
th.ese tests are not designed to provide data concerning whetlier c^r not absolute 
standards of competence or the acquisition of particular skills have been 
attained. 

Grade norms (i.e. scores based on group averages) from standardized 
tests are presently widely misinterpreted. It is widely believed by the public 
that norms from these tests define standards of competence for each grade, 
i.e. that they define an objectively determined level of performance on skills in 
a qiven subiect area tl\at all children in that grade should be able to reach, Ft 
is not generally understood that the "norming" process establishes averages 
requiring that 50 percent of all students be below grade level (i.e. average) 
when the test is standardized. The requirement that 50 percent of the stuc^cnls 
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iail beloNv and ^>0 percent above is unavoidable because the niolhodological 

appriKich of standardized testing is to ascertain a student's performance in 

relationship to the perforniaaco of other individuals on a ct)nimon scale based 

urjrjn the range of performance of the norminj^ group rather than lo measure 

s!udenl performance in relation to a set of defined standards. In accordance 

with the purpose of ranking individuals in relation to one another, test items 

are purposely selected to spread out test scores. When these tests are given 

to students across the State or nation, they function predictably to spread 
t 

students' toUil score out but do not show pupil performance h\ relation tt> a 
defined set of basic skills that constitute points on the continuum of ^earnin.u 
to master a specific subject. If the concern is to measure the extent to which 
New York State students are mastering basic skills (e.g. reading) then logic- 
would dictate that the evaluation system used should measure progress on 
each of those skills rather than on test questions that have been shown to have 
certain difficulty levels that demonstrate differences between individuals, and 
h£ive Ijeen selected and scored or scaled on the basis of a national or state 
sample population of students. 

Another problem associated with standardized tests is that they were 
not designed to measure adult standards. Some new testing methodologies 
in reading (and in scjme cases for other subject areas) are presently available 
to measure desired adult competencies: the National Assessment of Education 
Progress tests, Educational Testing Service Adult Reading Tasks, Harris 
Survey of ''SurvivaT' Requirements in Reading, etc. However, none of these 



tests has bet'n designed to incatjvir(^ [iroyrosb tovwurd adult reading conipetonc<; 
or lo lu'lp deteritune what pro^^ress seems to l^e appropriate in reaching those 
adult standards. Ideally, we would hcjpe to have some specification of an 
ultimate standard or standards indicating what skill levels an individual needs 
to function as an adult, P'urthe rmore^ tlie Department has concluded that an 
appropriate test should measure at any time during a student's educational 
exDerience the extent to which he or she is progressitig toward that stc\ndard 
or standards that is, a te.^t whicli indicates that, at some given time in the 
future, the student will be able to function according to the adult standards. 
Currently standardized tests do not allow educators to do either. 

Experts set standards for a grade level, but there is no assurance that 
their judgment is correct in terms of adult standards. For example, if a 
twelfth-grade student obt.iins a score that places him at the twelftli -grade 
norm, there is no conctasive d?la concerning his ability to cope successfully 
with the reading tasks that he will meet in the adult world. All that can be 
inferred from the test score itself is that his reading performance, compared 
with the performance of others in his age group, is about average on a set 
of test items subjectively selected as appropriate for his educational level. 
The items are included in the test because a certain percent of the students 
at that age were ablt; to master them wlien the test was piloted. 

Another major discouraging aspect of the review has to do with the in- 
ability of s tandardi^^ed tests to adequately measure progress within a school 
year and to provide teachers, principals, etc. with adequate information to 
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make immediate and meaningful adjustments in educational instructional pro- 
grams. In most cases, the items on standardized tests are too general or 
too few in number to serve as accurate measures of specific skill needs. 
Moreover, since the tasks used to measure general skills vary greatly from 
test to test, and even from level to level within a given series of tests, they 
are imprecise indicators of actual skill development and needs* 

In addition to the general concerns expressed above, there are a series 
of other specific misunderstandings related to interpreting grade equivalent 
scores that must be understood by readers of this report. 

Grade equivalent scores appear to be easy to understand and use. 
They have a ^'directness of meaning" that often causes them to be taken at 
''face value" without full consideration of the limitations imposed on them by 
procedures used in their derivation. 

Some of the limitations are obvious. A beginning third-grade pupil who 
obtains a 6. 0 grade equivalent score on a third-grade reading test does not 
read as well as the average sixth-grade pupil, nor should he be promoted to 
an average sixth-grade reading group or given average sixth-grade materials 
to read. His sixth-grade score (GE of 6.0) indicates that his reading ability 
is well above the average for third-grade pupils, probably because he has 
mastered the skills taught in his earlier grades in a more thorough fashion 
than mo3t pupils in the nationwide sample on whom the norms are based. 
This grade equivalent score of 6. 0, because it is scaled to reflect perform- 
ance average of sixth-grade students, would have quite a different meaning, 
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of courrio, if ihe pui)il iiaci taken a si \Lh radr readmi; test and, on that test, 
had o'jlaiacd a Gh'. score of 6. 0. rlius, ^racie equivalent scores of identical 
numerical \%'\lues do not indicate tlie same or identical levels of accomplish 
rnent and are not meaningful when considered in isolation from the contei\t 
of llie lest and the grou]) on which the norms are based. 

Grade equivalent scores have other less obvious limitations. Descrip- 
tions of these limitations are included in most lest and measurement books 
and articles, and therefore only a few pertinent ones are reviewed briefly 
in this report as reminders that grade equivalent scores are subject to 
misinterpretation and misuse. 

1. Grade equivalent scores may not be comparable from one lest ti> 
another in the same subject or in different subjects. Some tests distril)ute 
pupil scores over a wider ran^e of grade equivalent scores than do otlier 
tests. For example, a pupil wlio obtains a score at the 60th [)ercentile of 
sixth-grade pupils on each of two tests in the same subject (e.g. vocabulary) 
may obtain a grade equis^alent score of 6. 5 on one lest and a 7 ,0 or 7. 5 on 
the other. This is especially true of tests in different subjects, e.g. reading 
compared to mathematics, and is also true but to a more Hj-mted extent among 
different tests within the same subject, 

2. Grade equivalent scores assume that growth is uniforn'i through the 
school year. Because the scores for each n^onth are graphed or interpolated 
from the one or possibly two points during the year when the tests are stand- 
ardized, it is possible that a grade equivalent unit of one montli (. ii obtained 

at one time during the year does not represent the same amount of achievement 



as one attained, at another time during the year* Pupils tested at a time other 
than when a test was standardized may have actual acliievement slightly liighez' 
or lower than that indicated by their grade equivalent scores. 

Grade equivalents lack uniform meaning from low to high grades* 
A grade equivalent score one year below grade level at grade three n\ay re- 
present a serious disability (may be at the lOth percentile) while a score 
one year below norm at grade eight may not be considered lo rr?present a 
disability (might be at the 40th or 45th percentile, well within the average 
range). 

4, Grade equivalent scores are misleading when used as measures 
of growth, except for pupils whose scores are at or near the average for tlie 
norm group. The expected gain of one year in grade equivalent score for 
one year of instruction is not realistic for pupils or groups of pupils whose 
achievement is outside the average range, and particularly for those at the 
extremes of an achievement range. For a school with below-average pupils 
a one-year gain in its median grade equivalent (for example from 2.0 for 
pupils when in the third grade to 3,0 for these same pvipils when in the fourth 
grade) may well represent growth in achievement above normal expectation. 
Similarly, a one -year g^in in median grade equivalent for a scl\ool with 
above -average pupils may represent grow^th in achievement below^ normal 

expectation. ' 

f 

There are additional deficiencies of a more technical nature whicli are 
not discussed, here it is cleail thai che scales presently available to 
measure ijie effectivqnes s of instruction are notable to serve the purposes 
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often attribuU^d to llicni. ihcsi^ t;liOflcon«in^s oljvicuuily have confvising and 
distorting e flee ts on "faii"^" r<*pot'tt5 of the effectiveness of the educational 
system, a particular educational prui'^-ani, ur a component of an echu atio^ 
program. 

Tl^o review oi testing' niethodologies has led to a cK^ai-er idenlifjcatiL^n 
of properlics the Department desires for testing me tliodolop^ie ss U) be develope<i 
for the future. Progress la ({esigning tests wliich will have these properties is 
discussed in Chapter II. 

^ ' ^^State of the Art^^ Review - Availability of D isaggre^jat c d Inforn nation 
In order to answer the second question outlined on [)age 2 ^ >What 
types of instructional program s (or projects in the case o f cate g orical aid 
programs) are i lost effective for students of differen t char ade ri sties ? , a 
second major "state of the art" problem area has been examined below. 

The problem stems from the lack of information presently available to 
help niake decisions at various levels. In New York State it is estimated 
that there are 4, 15Z, 550 students enrolled in public and nonpublic scliools. 
In addition, there are approximately 4,400 school buildings, 149,000 in- 
struction! rooms, 182,000 classroom teachers, and an unknown nun^ber of 
instructional programs being used by teachers. The problem is to determine 
efficiently what has transpired at the classroom level or between classroonus, 
so that successful instructional programs can be identified and replicated 
on a wide scale, and so that resources can be reallocated, when required, 
based on continuous evaluation. At all levels of the educalional system-- 



local, state ov fode ral • -deci sion-rnake r a in particular have been limited in 
their ability to identify successful instructional programs or to improve 
instructional planning, management and evaluation practices because of 
the lack of pertinent information. 

llie lack of capacity to make such inforniation available has led to 
educational performance evaluations based on aggregate data of questionable 
relevance (e.g. gross expenditures by buildings, mean test scores by district 
and/or building, total numbers of students of different socioeconomic back- 
grounds in the district, etc, ). The evaluations based solely on these kinds 
of data have severely confused or damaged public attitudes towards tlie educa- 
tional system, since it has supposedly been shown that little seems to work 
in education and yet every parent reali/.es that their child is learning in 
schools. The state-of-the-art prcjblem relative to standardized tests further 
aggravates the problem in that standardized tests; 

'^often require skills or aptitudes tliat may be influenced to 
only a limited degree by experience in tlie classroom^ . .since 
standardized tests ordinarily have a low degree of overlap with 
the actual objectives of instruction at any given time or place. 
Standardized tests, except for some individually administered 
diagnostic instruments, <\xo designed to measure highly gener- 
alized skills. This ine\'itably means that scores on such tests 
have only a distant, even tenuous, relationship to the outcomes 
of tlie real-life instruction in a given situation. One cannot 
determine the effectiveness of educational programs or establish 
accountability in any meaningful sense if it is not clear that de- 
vices used to assess educational outcomes actually measure the 
objectives a given teacher, school, district or state is attempt- 
ing to accomplish ov^er ^ defined period of time. 
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Attempts to combine le»ss Lliaii aciequate uuUt^jne nieasuros and ag^^ ro).;.iti; 
information for evalual) pu:';.* -r. 1 is It^l [o sonu"^ mifair conclvisions re i: . ^ - 
ing [\\e cffeclivonotjs {)f cdvicational [ji v>g rams • This unsatisfactory situalioii 
has been compounded by the fact that there often may be as niuch variation 
between buildings within a district or even within a buildin^^ ov j^rade as be- 
tween any of the variables used for such analyses. 

Given the state of the art in respect to the testing techniques described 
above and the availability of disaggregated information, it is apparent that 
questions related to the extent to which basic skills are being mastered and 
to the quality of education programs cannot be satisfactorily answered using 
techniques that are presently employed on a widespread basis. Consequently, 
a new evaluation system (i.e. the system that has been referred to in this 
report as '^projected") has been designed and is in the process of being de- 
veloped and implemented as described in the material below. 

B. QUESTION #1; TO WHAT P^XTENT AI^E NEW Y ORK STA'PE'S 
ELEMEN TARY AND SECONDARY S TUDENTS SUCCESS F ULLY 
MASTERIN G BASIC SKILIjj/^ 

1 • Des i red Purposes of Te sts 

The first task the Department undertook to delertiiine a reliable 
answer to this question was to define w^hat types of information would l^c re- 
quired. Present efforts are focused on the development of new tests of read- 
ing ability (i.e. one basic skill) because of the importance of this skill. It is 
the Department's conclusion that tl^ere are three levels of testing required to 
measure this basic skill area (i.e. reading comprehension in the work done 
to date). 
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Qucsiioti il l: To what extent are New York Staters Elementary and Secondary 
students successfully mastering basic skills? 

First, there is a need to determine how well a student performs on 
reading tasks that will be encountered in adult life. Presently there is a great 
deal of criticism of the educational system by employers to the effect that 
graduates of the secondary schools cannot read materials which are required 
to allow satisfactory performance as employees. In addition, to function in 
this society, it is necessary that individuals be able to comprehend income 
tax forms, driver's license applications, insurance policies, etc, 

f 

There is a need, then, for a testing methodology to determine in^an 
efficient way (i.e. in away that can sample adult reading tasks and test 
students at low cost) whether or not a student can deal with required adult 
tasks. Obviously, different levels of acceptable performance on such tests 
will be indicated depending on each student's individual aspirations. For 
example, a student desiring to become a lawyer will have to meet a higher 
standard of reading ability than one desiring to enter occupations where the 
standards might be lower. 

A second level of testing is required in reading to measure progress 
toward adult standards over the entire school age range (i.e. beginning in 
primary grades, it is necessary to measure periodically to assess progress 
toward adult competence and to detect growth). The point is that, if the 
educational system waits until a student is close to high school graduation be- 
fore testing adult competency, it is often too late to compensate for any in- 
effectiveness of the educational programs taken by the student, or to deal 
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Question // 1 : T(]^\vhjit_e xleiij^ iu;;c^ 

iiludeats yuccessfully tnastcriug bawic skills? 

with other problexns, 1 iu* now Ic'sts closcribtMi in Lh(^ Inti'ocluction to t})i.s rcDin i 
attempt to measure adult competeni i (»s , but have not attempted to incorp^wa^e 
this accessary additional component* The Department feels that It is a ncdts- 
sity to do so in its research and development activities in reading. 

A necessary capability that is required for this second level of measure- 
ment is a technique that will allow reading materials to be scaled according 
to their difficulty. This will enable any material^ including tliose ve^^l by 
adults and considered to be requisite to basic reading competence, to be 
scaled and located at a specific point* Likewise^ reading neater ials typically 
encountered by students could also be scaled leading to the ability to equate 
student performance on the materials he uses to some poiiit or points on a 
scale where the difficulty levels of adult materials are also located. 

Using this technique students can be periodically measured during their 
school careers to determine their progress toward adult reading competence. 
Additionally, the scale will make it possible to determine the progress students 
and groups have made as a result of receiving instruction within a school year. 
Present measures such as standardized tests are not sensitive enough to 
determine reliable individual or group scores hetv/een fall and spring adn^inis 
trations of tests. This need is especially important in terms of makin.^ 
end-of-year evaluation, be it for categorical projects or regular instructions' 
programs. 
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Question //I: LO wlmt extent are New York Staters EIcmenLary anci Secondary 
s tudents successfully mastering basic skills? 

Furthermore, tlie scale will allow for coniparisons between instructional 
treatments, including both variations in objective sequencing, teaching 
strategies and materials leading to a determination of which provide greatest 
growth. With these kinds of data resource decisions could be made based on 
data generated from comparative analysis across various levels of. *:»c ucational 
luiits. 

The third level of testing requires a measurement device '.'lat, within 
a school year, can determine a student^s success in mastering those subskills 
which are necessary to reach a given reading level by the end of a school year 
and identify those sul^programs which are successful in reaU/^ing those 
subskills. Additionally, this level of testing is required to analyze the extent 
' to which the subprogram as characterized by particular sequences of sub- 
skills is leading toward mastery of those skills for students with different 
characteristics. It is the Department's intent to follow student performance 
and subprogram objectives over time with a computerized information system 
{described under Question ijZ) so that successful patterns can be identified. 

In summary, in the area of the reading comprehension, the Departn ie n^ 
has c oncluded that three levels of tests are required to fulfil l educational 
decision -making ne e ds> First, there is a need to determine how well a s ludent 
perfor ms on tasks th at m ay be encountered in adult life> S ec ond, the test(s) 
mu^^t be able to n\easu re p rogre ss toward adult competenc e ove r grades aiid 
bo ^.cnsilive enovgp to n\easure individual and pr ogram progress within 'i period 
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Q u e s ti o n H : T o what extent cxre New Yjj t'k S tate' s eleni enta ry and second a r y 
stvAdents successfully ma steving basic skills? 

such as a school year. Third > there is a ne ed for tests to verify t]\at spe cifi c 
subskills result in a given level of com petence at a s pecifi ed point in time, an (] 
to verify the most effective sequences in which subskill s can bo a cqui r ed^. 

The three levels of testing identified above for reading comprehension 
are thought to be equally applicable to other subject areas. 

It should be noted that neither the questions on or the description of past 
or planned work activities described below are directed at measuring the feel- 
ings, emotions, and attitudes that are acquired as a consequence c^f instruction 
and that are equally important as measuring basic skills, l^ocal district 

personnel have expressed a great deal of concern that the Department is not 
doing more in this important area. The Department realises the importance 
of developing adequate measures in tliis area but has concluded tliat, given 
the inadequacies of existing tests, the liniited funds currently available for 
this total evaluation prograrii, and the time it takes to develop measures 
of this type, that it is impractical to consider work activities in tliis area 
in the next few years. 

2 . Progress in Developing More Suitable Meas urement Devices 

The emphasis of the work completed ur in progress regarding the 
first two levels of testing needs (i.e. setting standards for and measuring 
progress toward adult competency) has been in the priority area of reading. 
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students successfully mastering basic skills? 



Present plans call for the development of similar tests in mathematics, if 

funds are available. Decisions related to other subject areas or to the 

measurement of attitudes, etc. have not yet been made. 

a. The Development of a New Test Methodology 

In the reading area, a one -year detailed research effort 
has been completed. The results of this effort were: 
I) a determination of the properties desired in a new 
testing methodology (i.e. the reading effectiventjss 
measure); 2) a description of the minimum number of 
broad development tasks which must be undertaken to 
develop the new reading effectiveness measure; 3) a re- 
view of existing approaches to the measurement of effec- 
tiveness in reading, both in terms of the properties de- 
sired and the technical problems associated with meeting 
the specified properties; and 4) a proposed work plan for 
developing the actual tests. The remainder of this section 
summarizes the results of this research effort under the 
four topic headings above. 

I) Properti es of the New Methodolog y; The properties 
identified in the report parallel the discussion above 
on the. purposes of the needed testing methodology. 
The specific properties include the following: 

a) A capability to quantitatively indicate th e reading 
difficulty of the broad range of adult rea ding 
materials. 

A capability to measure progress toward adult 
competence (i. e. , a standard or standards). The 
test must be able to measure the progress of 
individuah*? and groups toward being competent 
adults. 
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Q ue s ti on : 1\) vvliat extent are N^'Av^^^Mj^JSja^t^^^ 

s tudents successfu l 1 y ma s le rin^ baisic skills ? 

For t'xain[)le, in rearling il sliould: 

(I) Measure the ability lo cope with societal 
reading requirements imposed by law , 
sucli as compreliending incoiiic lax forms 
or driver^b license a|)[)licatU^'vs , and uMlh 
otlier materials intended by ^o\ o rniTiont 
agencies for the protection and woll-ijeing 
of citi /.ens, 

(Z) Measure the ability to read materials necessary 
to enter various vocations or professions, 

(31 Measure the ability to read materials that 
enable individuals to functio)i competently 
in their own behalf, sucli as advertisements, 
insurance policies, repair tnaiuials, etc^ 

c ) A capability to me asure individual ctfectivenes s . 
Since education is concerned with the develo])mcnt 
of individuals, the measure nuist yield reliable 
and valid individvial scores. 

d) A capability to measure system e ffectiveness. 
It must be possible to aggregate the scores of 
individuals (by grade, sex, ethnicity, etc) 

to determine how well the ed\icalional syslen^ 
is performing for different target groups in 
different schools, buildings, school districts, 
regions, and statewide.* 

^ ) A capability to measure growth in reading abilit y* 
The measure should be able to detect small chaiiges 
in reading ability, such as might be expected to 
occvir in one yearns time. Me^xsurement of gro\ip 
growth is an essential requirement of the measure* 
Measurement of individual growth, if feasible, 
is highly desirable* 
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Q ue s ti on H 1 ; To w hat extent are New York State's elemcMU ary and secondary 
students s uccessf ully niasteriir^ ba sic skills ? 



f) A capability to measure veacHn^ ^^^V^^^y _J^^^'__^^ 
entire school a ge range. Continuity of measure- 
ment, beginning in the primary grades, is neces- 
sary for measuring progress toward adult compe- 
tence and for detecting ^^rowth. Therefore, the 
measure should be applicable over all or nearly 
all of the public school age range. 

g) A capability to furnish rneani n gful scores . Scores 
on the measure should be readily and accurately 
understood by persons without technical knowledge 
of statistics or test construction procedures, such 
as parents, legislators, teachers, etc. There- 
fore, it must be possible to present scores in 
terms that are meaningful to such persons without 
sacrificing precision in reporting. 

h) A capability to measure specific instructiona l 
objectives. Work efforts associated with tlu? last 
property are described on pages 166-168. 

2} Minimum Number of Development Tasks Req uired, T]\e 
broad development tasks required to result in a new 
methodology to measure reading effectiveness are of two 
major types; PTrst, there are legislative -political tasks. 
In this regard it is important to state that the actual 
setting of acceptable adult standard(s) will under any 
circumstances be a policy decision requiring some 
subjective judgments but data can be made available 
which can make that task of policy-makers more 
rational. Scientists can and should contribute sound, 
impartial technical work to describe the range of 
reading difficulty associated with adult reading materials 
and also project wliat reading demands will be in say, 
15 years (i,e. when a student now entering school will 
graduate) but the judgn^ent as to what standard or 
standards are to be required must be done by govcnmu^nt. 
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Question ^1: To what ex ten! arc fji^v. ^^.rk SlaLc'ti *^J^'LI'l'il^'^ ^ ^'l^l-if^'-ir^ll^^^^ 
s tudcn t s iic^l: e s s f u 1 1 y n u us i o r i n ha sic hi k i 1 1 s 

'['he sccoiu! tyjje of broad <leveloj)inciil ta;i;k is 
categorized a sci cnLi L*i c - lechniea ] one, Tlie 
specific bi:ieiilliic ' leehnical taslvs to l>e underi .\kon 
include: (1) iiientifyiiig the vario\is kinds of materials 
tliat aduUs are called upon to read (i.e., definint^ wliat 
is called the corpus}; (?.) scaling ad'ilt r(u"idin;ijf tasks 
- because of the wide range of reading materials in 
existence it is important to sample tliem in an efficient 
manner and cletermine tlie clia racte ri sticf:^ (e.g. nun^jer 
of words, sentence length, luimber of commas , etc.) 
of passages or frequency of vise of words so that a 
formula can \)c developed to scale any t^ialerial effi- 
ciently; and (3) c:arry out the stops i^iecessary to test 
students (i.e. to des'elop tests) including such activities 
as selecting item formats, insuring that the test is 
reliable (i.e, that scores remain relatively stable upon 
relestiut^) and valid (i.e. does the test iueas\ire what it 
sets out to roeasure). 

B-SXij^'^^-Ji^-L-^^^ iL^^^^'S A }3p roaches. The review ^ff some of 
the newer lests being developed elsewiiere indicate that 
none of tliet^e tests nteasures indivichial or group groVv iii 
in reading achievenient or j^r ogress toward adult reading 
competence. in addition, there are teclmical deficiencies 
in the tests v^\en con^pared against the minimum recpii re- 
men ts contairicd in tlie i)re ceding pa rag f a pi i, 

'i)}e ii'searc]) report tlien discusses all of the tech- 
niceil alte r iia ti ves associated with dcvelof)ing a suitai)le 
measure, il w;is concluded tluit readability and, to a 
lesser exLOiit, vocabidary difficulty (i,e. incidence of 
rarely used \ crsus frequently used wor<Ls) are the 
properties to \)c measured. 

hj, ^iL^Pp^^^ i'Yi. w a s concl uded tlia t exi^-uhig te^Jls 
wi l l not yield I lie i f o 2^ m a ti o j i i' e q u i r ejjl t > a n s w e r t h e 
qu estio i\-- ' l o whcit extent are Ncj^;^,Yj';^]ll<; Sta*.e/ s 
cleme nj. a_rj^_ s t u d e n ts s_uc cessfuUy ni a c^H lu^ J^) a _s i c_j5^k![J1 s 
(i.e. reading in ll^e w^jrls^ underway];^" 
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Chi e H ti c) 1 1 // 1 : To wh at extent ar c New York State's elenienta ry and s econdary 
students succe ssfully mast ering ba sic skills? 

l^^'Qpo^ed Wo rk Plan. The following tasks are being 
undertaken in tlic immediate future to develop an 
appropriate testing n^ethodology : (1) a criterion scale 
of readability will be developed from adult and school 
age materials using a technique technically called the 
cloze procedure (i.e, where students and adults select 
words that have been deleted from passages); (2) an 
improved formula will be developed, since the wide 
range of passages that n^ust be scaled prohibit hand 
scoring of all passages (the formula will predict, 
based on the characteristics of any passage, what the 
readability is of that passage); (3) tests will be developed 
which test studentt^' and adults^ ability to read material 
of various difficulties. In addition, the coniputer will 
be used for sampling, for producing and scoring tests, 
and for reporting results, 

To continue the development work on tlie "residing 
effectiveness measure'" the Department lias submitteci 
a two-year proposal to the National Institute of Educa- 
tion (N, I, E. ). OtJier outside funding will also be sought 
this spring if NJ, E» funding is not secured, 

b. Other Uses for Data wliich will be Furnished by 
the New Test Methodology — 

There are significant uses of these data other than those for 
assessing pupil or program progress which are added in- 
centives for developing the "reading effectiveness measure". 
For example, if students were not meeting standards as 
measured by the new tests, it would be important that State 
and local educational decision-makers liave a way of identi- 
fying some of the factors contributing to the less -than-desired 
levels of effectiveness. Analyses such as those described 
below should contribute to an understanding of reasons why 
those standards are not being met» and indicate possible 
sources of corrective action. Perhaps all the factors con- 
tributing to a student's failure to attain such standards 
would not be identified, but significant information could 
be made available that would support constructive decision 
making. 



L' Design Concepts for a Measure of l^ffectiveness in Reading - A Feasibility 
Study, Riverside Research Institute, September 1973, pp, 179-1^/*^, 
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met rt'uld r.f i: iI'-z^m! i ii cor.juiji' (i ciii w- Ji infc' 

c oncL' i I' c . ■. I'll of nr I MJ.i'^^iis uwai^:^ nf^Ml. ' i-' ' a nchi r 

i»vo I' li.v' !\ > ':u .^rdar U) un< r s I J v. \ v .I'ty 'rU'^^^:'! 

» ! ■ . i^;-; by lilt" iv/vImIi .^lM':'^. Su«:'*i a i^hulv 

lion ^t,..p; /I !';at .d1 (l)lacl<, wliil". a ad Soanis'j- 

adult :{.\\[ri^ '^ir ^ra<h^ ten; ali (owci' rlass whi'c, 
l;l-;cl^,, .i . it - :> peaki nj; ) ^i tiu' ori i r nol retain 

^ )iiin:uir.iii ^: ; ^ai^ii: i^y radii i^vtJvo/ l.i 1 i ^:^ fuith^'r 

by e c j L ] u : : ■ w t : 1 1'> e a v 1 1 a c h o ol y t.' a v r o[\^ i\ v a nv] c 
ihia-yir^li i\v: rv\A Llioii lovcled oft' so (hal Hiv imU> ol 
^ r c^w t hi i i ; ' ; I . ! u t cl r o a si i n g skill w a ) t : f « ^ r g r <\ ci e s j \ 
throiii-di \\:^ \\r 1luui i\n' L^ractes oiU' thr<)M|;li fivi^; fiirtiier- 
r'nr^rt:, Ijy u:ati^ j;. racle, dci facto tandii fli^ lias'ir >**iat'lK'<l 
llio lo\ ^'l >a a Jiilt y (a I'ula rdfJ , ( 'on Li nt;ui|.^ vvilli cuir hvpo- 
tluM- .'J : pi ItM i:t? a>^>ni:iie ihul ■)]) middle <daii^> 

1.1 idt' ■ ) ; ;■ [:.-\. Ill' t' 'vVit. t'lk d i;i*adf,' :;ia^dii"d; i li 

g V n d i , t ) ■ 1 1 \ . I • , I'l i J p; in r a d t- t h i o , ^ j 1 ] h j w c t 
cla^- . : I M . iwl alhdii i^rarlo U'v.d s 1.5 ; i diU'd s . iv 
then Tiill . li.a ilvv^j'ild Ijj.; l'^- a oi; u hi e iiuputa' in!o 
Uic i\' J- 1 i . 1 1 ^ 1 'v t.f rt:tl acini;, f!>r Io'aoi' < Icvs^; t>t\idcMLs, 
the L'ciio i;y ' dji^li I lie .slandai'ds inc:i'ea^i!d in pp'adet; 
Ihi'tMiv;:: 'i'/r^ .1 1 c r c a id 1 i ^ (lie rate in grarjes six tlu'oui^ji 
Lu'clvc, Uid ..^'idnf' the f^resciil dc^ iacto c i"a<lc' ton 
staadird i'iC txru'ctcd \^ilue in t:^rar|e uvclvc, Sticii 
cli irp:^ . rca.'oniljly he c t > n i d vM xmI in i^rdcr lo 

d^Ji h>v/er t la.iS studi-iU: aU-niiK'd adult 
s t-n^id^: a - ■ ' ; i" " L ■ i ' i 
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Quastion #lj To what extent are New York State's elementary and secondary 
students successfully mastering basic skills? 



2) Analyses of instructional materials . The reading effec- 
tiveness measure could be used in analyses of instruL- 
tional materials designed to locate possible problems in 
the readability and vocabulary learning demands placed 
on students. Once such problems were identified, re- 
sponses designed to alleviate them, and hence to increase 
effectiveness, could be suggested to senior managers 
of the State Education Department and to local educational 
decision-makers. 

The analyses that are illustrated below do not, by 
any means, exhaust the analyses of instructional mater- 
ials that might be carried out in looking for sources of 
failure to meet adult standards. For example, the 
question of the consequences of alternative methods of 
teaching reading on attaining standards is not touched 
upon. Rather, the illustrations are limited to analyses 
that are related to the readability and word familiarity 
measures (i.e. subcomponents of the reading effective- 



ness measure — 

a) Disorder in instructional materials . The 
vocabulary of instructional materials is 
presently not set in any systematic sequential 
manner. 

Evidence presented by Stauffer (1966) indicates 
that reading programs differ concerning which 
words are introduced in which grades. If there 
are major differences between the vocabulary 
in various sets of instructional materials, there 
should be difficulty in maintaining a logical se- 
quence of instruction. Whenever a school changed 
books or whenever a student changed schools, 
students would be apt to encounter a great number 
of words not previously learned or, alternatively, 
be asked to learn words already mastered. The 
problem of encountering a great number of new 
words could be expected to be most serious for 
those students who are most dependent on the 
schools for what they learn, presumably educa- 
tionally disadvantaged students. 
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Question §1; To what extent are New York State's elementary and secon dary 
students successfully mastering basic skills? 



On the hypothesis that one knew what materials 
were used in all grades, the data would be avail- 
able with which to determine the degree of simil- 
arity across reading program materials from 
different publishers with respect to the vocabulary 
and readability introduced in each of the grades. 
If a display of the overlap of readability and of 
vocabulary across programs confirms that, in 
fact, there is appreciable chaos (i, e» , little 
overlap), corrective action would be indicated, 
A logical step to correct the problem would be 
for the State Education Department to direct 
publishers that there be {at least) a minimvmi 
specified amount of overlap in vocabulary and 
readability across reading programs. 

b) The effect of the amount o£ learni n g expected per 
time unit (load) on reading development . There 
is evidence to suggest that the vocabulary load 
that is being placed on students is not being 
carefully planned or controlled. First, there is 
evidence suggesting that publishers do not co- 
ordinate vocabvilary across cvirriculum areas. 
Stauffer's (1966^ comparison (3f reading, arith- 
metic, health, and science books showed little: 
overlap in each grade bt;tween tVie vocabulary 
words introduced in the books used in the 
different subject areas of the curriculum. For 
example, he found that while 2153 new words 
were introduced in seven reading series in the 
third grade, and 2150 new words were introduced 
in three arithmetic series in that grade, only 
421 words were common to both lists. Moreover, 
many words appeared in textbooks in different 
subject areas which did not appear in any of the 
seven reading series at any grade level. Stauffer 
has estimated that even if a student somehow had 
the opportunity to learn the vocabulary of all 
seven reading series, he would learn only half 
the words he would encounter in his arithmetic 
books , 
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Question // 1 r 



T o what exte n t are New York State's elemen ta ry and secondary 
s tudents successfully mastering basic skills? 
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Table 42 



Grade Level Sample for Participating Districts 



Grade Sample Test Test 

District > Levels Size Levels Forms 

Brentwood^ 4-5-6 1000 5 20 

District 28^ 4-5-6 400 6 30 

Greece 4-5-6 1500 3 IS 

Jamosville DeWitt 4-5-6 1000 6 30 

Plattsburgh 4-5 400 4-5 20 

Rush Henrietta 4-5-6 500 6 30 

Syracuse^ 4-5-6 2500 6-7 30 

Utica^ 4-5-6 1200 5-6 30 

West Seneca 5-6 800 2 15 

Yonkersa 4-5-6 700 5-6 30 

Totals 10» 000 250 



Starting 
Date (73-74) 

Sept 

Feb 

Jan 

Jan 

Feb 

Feb 

Feb 

Feb 

Sept 

Jan 



These districts have a common standardized achieven^ent test (CAT) and 
will be combined for some analyses in the study* 



Figure 1 shows the major activities and schedule of 
the study. Districts selected objectives in the areas of 
reading comprehension and vocabulary using the newly 
available comprehensive New York State bank of reading 
objectives. The objectives in each of these areas were 
''leveled'^ into the range of reading levels found among 
students in grades 4-5-6 in a district. Lists of reading 
materials and word lists were then associated with the 
reading objectives in each level. These and other data 
constituted the specifications and raw material used for 
generating some 750 to 1200 test items for the criterion- 
referenced test forms for each district. Typically, each 
district will have 5-^6 test forms at each level, with 30-40 
test items (half in the area of vocabulary and half in the 
area of comprehension) on a form. There are, then, 
some I50'-250 test items available for each level of testing. 
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Q Hostility li I : H'o vvlutl cxlent are New V orkvS tate'ri elem entary a nd se<.^ondary 
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Figure 1 

Schedule of Study V^ctivities (1973-74) 

Activity 

Selection of Objectives 
Prepare CRT Test 

Specifications 
Develop Tost Forms 
Administer CRT*s 
Administer Standardized 

Tests 
Return CRT Reports 
Analyse Data 
Prepare Report 

PEP test collect from the sixth grade sample. 

Administration of adult reading competency measure developed by ETS 
to 6th graders only, 

^*A variety of different standardized tests scores will be available, some 
with only post test scores, whereas others will have as many as three sets of 
scores for the school year. 
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X 

X X 
X 

X 



XX XX 

XX XX 

X X 



The administration of the c ri te r i on-- refe renced tests 
shown in Figure 1 begins in January of 1974. Students 
in a given classroom will receive the test forms on the 
appropriate levels in random order on a monthly basis 
until they have received all five forms in a level. All 
levels of tests developed for a district may be used in 
a classroom contrasting sharply with the typical admin- 
istration of a single level of a standardized test in all 
classrooms in a grade. 

The remaining elements of the schedule shown in 
Figure 1 indicate the return of classroon\ and district 
level reports, on a monthly basis, using the data pro- 
cessing system called CAiM {Comprehensive Achievement 
Monitoring) described under the Instructional Evaluation 
System section of this report. Also shown arc the analytical 
and reporting stages of the project. 
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Q u <» s ♦ i ( H 1 III : '['owhat cxtontare New York St ale's ele mentary and sec ondary 
students su ccessfviUy mastering basic skil ls'^ 

The stud y has alre a dy yielded some in ^portant data 
un the feaaibility o{ vising c r i te rio!) - refe renced Ic s 
on a statewide basis , 'Vhai is, it is obvious at t his p oint 
that ihc participatin g districts hav e largel y selected the 
aan'ic instructional objectives in the areas of vocabula vy 
and comprehension^ Where the districts generally differ 
most is in Ihe ^'content''' associated with the ob jective s and 
in t he numb er of t esting levels a]>plicable to the children 
in a given classroo ni or buildi ng. Urban distric ts tend to 
u se the full range of test levels that can be m ade available 
and also add one more lower level of tes ting compared 
Willi suburban o r advantaged districts. 

This experience with creating several levels of 
criterion- referenced tests for a district suggests that 
some 15-20 levels of testing could be created to cover 
the full range of reading levels generally demonstrable 
in grades K-6. An acceptable plan for evaluating programs 
in districts would allow for assigning each individual student 
to the appropriate developmental test level, regartlless of 
grade level, A district may then assess the effectiveness 
of its programs in relation to the progress of students 
from level to level. 

T he technology created to support the developm ent (>[ 
criterion^refcrcnced tests in reading has so far pr oved 
ba sically adequate for the task. The bank of reading ob- 
jec lives is currently being computeri/.ed and made availaljle 
statewide. Test items for the development of criter ion- 
referenced tests in reading will a l so be ind exed by computer 
to support ec o nomical test development on a much l arger 
scale i n the fvUure, As experience in criterion-referenced 
test developm ent continues to be gained in the study, the 
abil ity to support this mode of testing through increased use 
of technology a ppears more and more feasible . 

In 1974-75^ this study will be expanded in the same 
districts to cover grades 1-6 in reading. The statewide 
objective and test item bank will be expanded. In 1975-76, 
packaged materials for use in all districts will be completed, 

^n svmimary, the question i o what extent ar e New York 
Sta ters s tudents s uccess fully n astcring l^asic s kills can he 
answered in different v ays depending on how "basic skills" 
are defined. this projected" program three levels are 
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seeti as being required. Pr<^gresB toward developing suitable 
measures is progressing well on 'interim" activities planned 
ur iniplenicnlofl to best answer tlie question in the interim 
period, A more complete review o£ the "interim'^ prograni 
was discussed in Chapter II. 

C. QUESTION U: WHAT TYPES OF INSTRUCTIONAL 
PROGRAMS (OR PROJECTS IN THE CASE OF 
CATEGORICAL AID PROGRAMS) ARE MOST EFFECTIVE 
FOR STUDENTS OF DIFFERENT CHARACTE RISTICS? 

The Department's efforts described in Section II A have a direct relation- 
ship to this question^ Education must have adequate measures of basic skills 
before comparing alternative approaches (e. , instructional programs or 
other "input'' quantities) to try to respond to this question. In addition to re- 
solving the testing problem, the Department is convinced that improved methods 
are required to obtain explicit information about programs in order to be able 
to identify which instructional programs work better than others. 

At present, as indicated in the introduction, it is not feasible to trace 
comprehensively uliat approaches are being used in various districts, buildings 
and classrooms throughout New York State, The practical problems inhibiting 
the collection of meaningful information have forced educational evaluators to 
use a variety of aggregated information. Even in the case of categorical 
projects funded under the Staters Urban Education Act and E. S. E. A, Title ], 
where there is considerable more documented information on what is supposed 
to happen if projects are funded, seldom is there explicit information on what 
instructional activities will be used. In addition, there generally is no prac- 
tical method of tracing whether or not project elements were implemented as 
proposed and whether or not some or none of the parts of the instructional 
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v^iu'Slitui '/o: Wluil lypcH of instructional progranis (or projects in tht^ case of 

r<t U'ij.() rtc a 1 ^^i<i ^^[^'HJi''^'^^^ ) arc n^ost cffccMvo for s tudents of diffe re nt cha racto ris tics ? 

prot^rani wore suc<.:ossful for students of different characteristics. IniproviriK* 
tins situation is equally as in^portant as developint' more useful testing method- 
ologies. 

Eliminating these practical problems is a tren^endous undertaking i^iven 
the niuiiber of scl^oo) districts, buildings, subjects taught, classrooms, and 
students in the State, The Department Jiow feels that this can Ije done by 
gradually iu\plementin^ the Instructional Evaluation Systexi^ (I.E.S, ). 

The I.FJ.S. is now being implemented with t\S'o different coniiji ter .'^^oft- 
ware systems --one called "C, A.M." (Comprehensive Achievement xMom tor .i^ ^ 
and the other called "I. S. S. " (Instructional Support Sys tern ). A decision was 
made some three or four years ago to develop these two systems bt-cause, 
even though the I.E.S. concepts were generally accepted as being needed, 
no ojie really knew how best to ope ra tionalize them, in particular in terms 
of the coniputer software system requirements- 

vSurprisingly, both systems, although developed and pilot tested inde- 
()endently, are very similar. With the growing acceptance of both a ppr (jacht^ s 
m an increasing nun^ber of districts, it is clear that the most prac;tu il course 
of action is to vise both because the resources are hard pressed in both lases 
to serve the whole State in the next few years. Generally the C\ A« M. system 
is being implemented in upstate and Long Island districts and the I. S. S. in 
New York City, The I.S.S., however, was piloted in an upstate district. 
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Qiiostioji if2,: What types of inslructional [)rog ranis (or projects in the case of 
cateu.cjrical aid prt^^ran^a) are most effective for students of different cliaracter- 

i6jLij^\s / 

It sho\ild be noted that the Department believes thtit the inapleuientatii>n 
of the Instructional Evaluation System isalso required to improve instructional 
olannini;, management and evaluation pmctices in local school districts-- 
i.e. to answer question ^3. The description which follows of the concepts of 
the Instructional Evaluation System should make it clear that this reasoning 
is sound. 

I y The Ins tructional Evaluation System - Overview of P urposes 

The Department believes that standards relating to the following 
aspects of the instructional environment need to be established and met if 
Question ^2 is to be answered. As indicated below, all of tlicse standards 
are not now being met. 

1) quality of instructional objectives; 

2) quality of instructional program designs (i,e, , well-designed 
and documented instructional programs); 

3) appronriate test ins truments selected to measure perf onioance ; and 

4) utilisation of information froni the instructional program for 
within -school-year and between*school-year decision-making. 

a. Instructional objectives 

Objectives must be well foriiiulaled and must, be related to the 
measurement tools selected and vice versa. Over the past 
several years, program objectives have been improved in 
categorical project applications and also in regular school 
prog ranis. Th ree additional improvements, howcve r, are 
required in many disti'icts. 
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Question ^^2* What types of instnictional programs (or projects in the case 
of categorical aid programs) are n^ost effective for students of different 
cha racteristics 



1) Objectives continue to be stated for the whole program 
(end-of-year program objectives) rather than for 
portions of the program (e, g, , t\vo-week groups of 
learning activities that are generally called modules). 
This present reliance on end-of -program objectives 
precludes inalysis of parts of programs. For example, 
a program might not be too impressive in terms of the 
final results obtained, but many components of the 
program might well be effective. Documentation of 
module objectives is also needed if continuous program 
adjustments are to be made while the program is in 
progress rather than when the end-of-year evaluation 
has been completed. 



2) Too many project objectives are stated in ternis of 
results expected from standardized tests (e, g, , the 
participants will advance from the lOth to the 2Sth 
percentile on the test). These types of objectives 
are popular because standardized tests are readily 
available. An analysis of some instructional programs 
indicates that these tests are frequently unrelated, to 
varying degrees, to the subject material being taught. 
Therefore^ this combination of objective type and the 
test instrument employed has frequently been found to 
be inappropriate for the evaluation of instructional 
programs. ( This barrier was discussed in the Intro* 
due lion. ) 



The Department is assisting local districts in 
developing district-made mastery tests which are 
closely related to the objectives of each instructional 
program. Results of standardized tests are also 
being recorded to analyze the relationships between 
the two testing methods until replacement tests (i.e. , 
tests like the Reading Effectiveness Measure) are 
developed. 
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Question f^Z: What typos of instructional prot^ranKs (or projects in the 
case elf categorical aid programs) are nu>sl ei'feclive for stutkMUs of 
clit'i'e rent charucto rislics V* 



3) The cxistinjj; practice is to clocumenl ol)jeclivcs for 
categorical programs only, and not. (or regular 
ongoing programs. The ii"n[>act of a categorical 
program cannot be properly determined unless 
comparable information about regular, ongoing 
programs given in the san\e school district is 
available. In those districts where the full IKS 
is being implemcnled, the DeparUi^ent is assisting 
districts in developing module objectives related 
to instructional activities for all progran^s. '^hcse 
objectives are not slated in terms of expected ^^t^^- 
dardized test results, but instead are stated in 
terms of the behaviors to be developed as a result 
of instruction which are in turn assessed by criterion 
referenced mastery tests. 

In districts where the full IKS is being in'iple- 
mented, the Department is supplying teclinical 
assistance to insure that objectives are properly 
defined. In these districts, a computeri/.ed 
support system is provided to efficiently store 
the objectives for management and evaluative pur- 
poses. 

b. Instructional Program Designs 

The second standard is the quality of ))ro^ram design 
(i.e. , liaving a well^designed and documented instruc- 
tional program). Progress has been made in having 
districts define their programs, bvit learning activities 
grouped into modules must be more fully documented 
if the local districts or the State are to know to what 
extent particular motiules were effective in teaching 
different groups of students. Currently, most school 
districts do not have the necessary compute ri/.cd 
capability for documenting instructional programs 
efficiently, either for regulai^ or categorical programs. 
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lesiion III: What typcvS of inslructional progran\a (or projects in 
^ case of categorical aid prograius) are most effective for students 
diffe rent characte ris tics 



In districts where the lElS is being irriplen^ented, M\c 
con^puterized support system is available for storing 
progran^ information for managenAent and evakiative 
purposes. The Department is rendering technical assist- 
ance to assist pilot districts in developing modules and 
objectives (i,e. , learning activities grouped according 
to specific objectives and covering roughly two-week 
periods of instruction) for the entire educational program 
(c, g. mathematics). 

c. Utilizing Information for Decision-making (i.e* feedback) 

Currently, neither teachers nor principals receive timely 
information that would permit adjustments to be made within 
and between classrooms. This situation arises because it 
is physically impossible to manually process all the neces- 
sary information on instructional objectives, program 
modules, module tests, end-of-semester and end-of-year 
tests for all students and. all subject areas. The computer- 
ized support system can make all this information readily 
available. With this capability, principals would find it 
possible to assume a larger role in instructional manage- 
ment and evaluation* 

In those districts in which the lES is being implemented, 
the Department is rendering technical assistance to teachers 
and principals in assisting them in utilizing information for 
decision-making- With the lES, it is possible to make 
program adjustments before students have progressed to 
a point where intervention becomes very difficult. 

With the implementation of*the I. E. System in more 
districts, the Education Department will receive much 
more timely information, and reporting to the Governor 
and the Legislature can be improved because of the capa- 
bility provided by the computerized support system. 



Question ^Z: What lypes of instrvictiunai jirograma (or projects in 
the case of categorical aid programs) arc tnc^st effective for atudeiUs 
of different characteristicH 



The iniplenienLation of the hE.S» a^i described briefly 
above obviously will allow both local and State decision- 
makers to answer question ^2--What types of instructional 
programs (or projects in the case of categorical aid 
programs) are most effective for studeiUs of different 
characteristics ? 

Since the Department would have access to ])rogram 
and module information on a continuous basis in those 
districts which have implemented the full IKS, the capability 
to determine the comparative effectiveness of modules 
and/or of total programs for the disadvantaged would exist. 
Information on successful programs can be disseminated, 
and local districts can be warned if some approach had 
failed elsewhere* 

d. Computeri^ed Support System 

Because the implementation of the I, E. System is strongly 
dependent on the computer support systen^ mentioned 
above, a summary of the system which bay i^een developed 
and implemented is presented next. 

The Nevv York State PMvication l^epa rtmijnl l)as iiccpu retl 
rights to a flexible software system which has unicpu; features 
(!• S. S. ). The system will allow local education agencies 
(LEA^s) to build pupil and instructional j^rograni files as 
part of ongoing administrative and iustructLoti^tl activities. 
With this system, disti*icts can record comprehensive 
information on their instructional programs as well as 
information needed for administrative and reporting purposes 
(e.g., attendance, scheduling, transcriptSj and reports to 
the community, parents, students, tlie State, and tlie Federal 
government). Presently, to fulfill reporling requirements, 
most districts must either manually lift data from many 
paper filing systems or ask teachers to provide the infornia- 
tion. In either case the burden is nriuch too great, espe- 
cially for teachers. Furthermore, such "survey " proccclures 
to recover manually filed data are inefficient, and may 
result in inaccurate reporting. 
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Question #2: What types of instructional, programs for projects in the case of cate- 
);ori cal aid programs) are most effective for students oi different c haractcristics V 

The unique features of the computer support system which 

thi» nepartU'cnl is irriplemf^nti ng for lES include: 

capability to handle extensive amounts of data 
on students, and programs at a reasonable cost; 

capability to add or delete elements of information 
at will without redesigning tlie system {thus users 
are not locked into fixed data files}: 

capability to produce a variety of reports (users 
can change the format and content of reports and 
"massage" data from many files at will); 

ease of use by laymen {a layman's guide is ?)rovided 
to enable users to address the computers without 
special training); 

capability to enter data easily as part of ongoing 
ope rations* ^ 

e. Summary of the KE.S. Components 

■ The UE>S,> in sun\mary> consists of two principal components * 
First is the computerized information-handling component 
whic h scores tests^ keeps records on pupil and programs 
and generates reports for students, parents, teachers and 
e ducational decision-make rs« The State can gather data from 
local school districts on successful educational programs . 
It is important to emphasize the State will not receive infor- 
mation on specific students. It is only interested in program- 
matic information . 

The second component of the L. E. System is a set of 
procedures at the lecal level for ; 

1 ) systematicaii y fleve loping instructional prog ram 

objectives that can be operationalized across classrooms ; 

2 ) systematica l ly developing or assembl i ng instructional 
activities and master y Lest i tems linked to instructional 
program objectives ; 
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Queation What types of ins Lnnjlional programs (or projects in the case 

of categorical aid programs) are most effective for students of different 
charactc r i s tic s ? 



^) systematically measuring student pe rformance vvi th_ 
^^\^,^ter y tests , and interpreting test scor e s us i njj 
predetermined pe rformance cr it eria; ^ 

^ ) systematically reaj locating Jlxis true tio_nal re sou rcos 
Is ! * y^}^M}' teach er time) ^ tq im prove^ s tucl eiU 
performance; and. 

5 ) sys tem^a t icallv recording in structiona l a cti^vit,i,es, 
objectives ^ and test sc ore s to provide a basis for 
upgrading ins t ructiona l program content and ev alu- 
ating inst r uctional performance oy e r tinic« 

T hese two c omponent s Jf unction tojt^e tjie r to provide feed - 
bac k inf o r m a tion to teac he r^ ,£i}d ot^ he r educa t i on cU_d e c i s ion - 
makers s o that student performance can^ Ije inipi'oyed by ; 

detjc tin^ disc repancies be tw e e n_p laji ne d^ a n d 
actual perfonimnce l^y students 

takin g inmiediate cor rective action witJi respect 

to instruct i onal resourc es beiri ^i; a llocated to and 

in structiona l activities being implement e d in tlie 
clas s roo m 

ta k i nji l ong - te r rii c or re c ti ve iLS^ ^i^ALJ^^i^b r espe ct 

the supp ly of instruct iona l t'esj) iirces and 
a c ti vi ties n: i ade availab le to clas s r of2,ni .tea^he r^s. 

The feedback is represented on the accompanying diagram. 
One starts out with decisions on resources such as who will 
teach whom, what rooms will be used, what instructional 
activities a.re to be used, etc. and then traces general re- 
sources and instructional activit^\iS as they are used. After 
classroom activities are implemented, tests are given and 
continuous information is made available for a number of 
purposes such as trying alternative learning activities if 
the origixial oiie fails with a particular Lyf)e of student, 
documenting successful approaches, utilizing reading 
specialist's time more efficiently and effectively and for 
many other decision -making purposes. In addition, reports 
to parents and the community can be made. The feedback 

loop provides LE'.A's with information that has not been 
available in the past, 

O 
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Queslion fiZ: What types of intstvuclional programs (or proj(!cts in 
the case of categorical aid programs) are most effective for students 
of different cha r a c te r i s ti c s ? 



f. KvaUuition of l.E»S. -leased Instrviclioiial Proj^rams 

The question of whether the implementation of an I. E* 
System in a school district n^akes a difference in terms 
of stucJent performance is difficiilt to assess since 
standardized tests are the only nieasuroment device 
currently available U> measure sv\ccess. This report 
has already indicated that its sensitivity to actual 
classroom instruction is qviestionable. Even with this 
constraint, a pilot district in vipslate New York has used 
standardized test results as a measure of success be- 
cause it felt (and the Pepartment concurred^ that an 
evaluation of the system was required. Thi.^ evaluation 
is not completed but it is presented here to indicate some 
measure of the effect of implementing the I.S. System. 
The complete evaluation will be available in ap|jro\imately 
six months. 

Data available at this time include the results of the 
California Achievement Tests in reading unci mathematics 
administered across grades, to different students in each 
grade, before aru after the installation (^f ISS, e, grade Z 
before ISS and grade 2 after [SS* Tlic results of these tests 
are shown in Tables 43, 44, and 45. White the data presented 
have not been subjected to statistical analysis their, implica- 
tions are especially significant when related to llie manner 
and intensity with which fSS was implemented* 

Throughout 1972-73, implementation was stressed 
most heavily in the primary grades, next in the inter- 
mediate grades and lastly, in the high school. It is 
interesting to note that the data shown in Tables 43 and 
44 reflect these activities in that tlie trend in gains or 
losses are consistently positive in the primary grades 
but appear mixed in tlie intermediate school grades. 
The significance of this finding sliouUrnot go unattended. 
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Table 43 

Mean Reading Achievement Before and After ISS in Guilderland Cantral Schools 

Before iSS Means After ISS Means 

Trend 







Expanded 


Converted 


Expanded 


Converted 


+ Gc 


Test 


Grade 


Spore 


Grade Pt. 


Score 


Grade Pt . 


- Lc 








Primary 


Grades 






Vocabulary 


2 


367.8 


3.3 


373.1 


3.6 


+ 


Comprehension 


2 


382.0 


3.3 


390.4 


3.5 


+ 


Total 


2 


360 .7 


3.3 


368.4 


3.6 




Vocabulary 


3 


404 .2 


4.6 


409.5 


4.6 


+ 


Comprehension 


3 


430.8 


4.5 


445.1 


5.2 


+ 


Total 


3 


408.8 


4.6 


421.8 


5.1 


+ 


Vocabulary 


4 


426.2 


5.3 


432.3 


5.5 


+ 


Comprehension 


4 


468 . 5 


5.5 


467.6 


5 . 5 




Total 


4 


437.0 


5.5 


440.1 


5.6 


+ 


Vocabulary 


5 


463.0 


6.6 


469.6 


6.8 


+ 


Comprehension 


5 


496.2 


7.1 


506.6 


7.3 


+ 


Total 


5 


471.9 


6.8 


480.7 


7.0 










Intermediate School 






Vocabulary 


6 


498.1 


7.5 


501.6 


7.7 


+ 


Comprehension 


6 


512.3 


7.6 


514.6 


7.6 


+ 


Total 


6 


500.0 


7.6 


503.5 


7.7 


+ 


Vocabulary 


7 


536.6 


8.6 


533.1 


8.6 




Comprehension 


7 


560.2 


9.5 


550.7 


9.1 




Total 


7 


546 .2 


9.0 


538.7 


8.8 




Vocabulary 


8 


577.3 


10.1 


579.2 


10.1 


+ 


Comprehension 


8 


585.0 


10.3 


597.8 


10.6 




Total 


8 


580.0 


10.1 


. 589.5. 


10.4 


+ 



■ s> 
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Table 44 



Mean Mathematics Achievement in Guilderland Central Schools 



Before ISS Means After ISS Means 

^ Trend 

Expanded Converted Expanded Converted + Gain 

Test Grade Score Grade Pt. Score Grade Pt. - Loss 









Primary 


Grades 






r^r> rn rill t* A i~ i n 


2 


4 


3.2 


313.5 






Concepts 


2 


358.9 


3.4 


363.6 


3.6 


4- 


Total 


2 


315.1 


3.3 


319.4 


3.4 


+ 


Pnmoii f* «=5 ^ 1 rj n 

v.* U C4 ^ X Kj 1 1 


*^ 




4.5 


363.2 


1 . ^ 


+ 


Concepts 


3 


402.8 


4.9 


412.1 


5.2 


+ 


Total 


3 


368.7 


4.7 


373.6 


4.8 






4 


1Q7 S 


5.2 


400.9 




•4- 


Concepts 


4 


429.5 


5.6 


434 .2 


5.9 




Total 


4 


398.2 


5.3 


402.2 


5.4 


+ 


Computation 


5 


434.4 


6.0 


440.2 


6.1 


+ 


•**OllC tip Ub 




d "5 9 


6.9 


470.6 




4. 


Total 


5 


434.4 


6.2 


441.2 


6.4 


+ 








Intermediate School 






Computation 


6 


463.0 


6.6 


484 .4 


7.3 


+ 


Concepts 


6 


485.6 


7.4 


488.4 


7.6 


+ 


Total 


6 


467.7 


7.1 


480.7 


7.3 




Computation 


7 


515.3 


8.1 


538.3 


8.5 




Concepts 


7 


545.4 


9.0 


551.4 


9.3 


+ 


Total 


7 


526.0 


8.5 


541.0 


8.9 




Computation 


8 


562.9 


9.4 


576.4 


9.7 


4 


Concepts 


8 


601.9 


10. 7 


591.1 


10.4 




Total 


8 


580.0 


10.0 


581.6 


10.0 






s 




* 




? 


i 

¥ 

9 

I 
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Table 45 



Percent of Students Achieving Below 
Aptitude in Reading and Mathematics^ 



Academic Year 


Reading 


Math 


^ 1971-1972 
Before ISS 


9.0% 


9.0% 


1972-1973 
After ISS 


6.8% 


8.1% 



^'Below aptitude" is defined as a meaningful 
negative difference between a student's achieve- 
ment score and his or her expected achievement 
score based upon IQ. 
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Question i^Z: What types of instructional progriims (or projects ij-i 
the case of categorical aid programs} are most effective for students 
of d i t' (v re n t c ha r a c te r i s ti c s 



The installation of the ISvS in an upstate district was 
unique in many respects, including the very important 
fact that the systetn is totally dependent on classroom 
teachers for the development of materials and definition 
of instructional strategies. In contrast, other attempts 
to define and implement such systems have relied heavily 
on predesigned software and control by outside consultants, 
P^vperience has shown that intimate teacher involvement 
is a prerequisite for success in such endeavors and these 
data support that conclusion. They also support the con- 
tention that unless one is prepared to do something in- 
tensively it slxouldn't be done at alL 

Table 45 shows the Percent of Students Achieving Below 
Aptitude in Reading and Mathematics before ISS and after 
ISS, across all grades. It can be seen thai, before the 
introduction of ISS, 9. 0 percent of the students could be 
classified as ''below aptitude" in reading; after ISS, only 
6. 8 percent were counted in that category. In mathematics, 
the figures were 9. 0 percent before and 8. 1 percent after. 
In both instances we again see a trend toward positive 
improvement. 

While the data presented are, admittedly, preliminary, 
it does seem reasonable to conclude that the I,S. System 
shows definite signs of success. 

Z . Progress in Implementing the Instructional Evaluation System 

As indicated in the evaluation just discussed, one upstate school 

district has over the past three years implemented the I.S. System. This 

project ,as supported by federal funds« fliis funding will terminate this 

year but the school board committed itself some three years ago to continue 

it if results were positive. It appears that it will be continued. 

- ,■://■- , 

r^n the present year the highest priority is to begin worki?Tfg with New York 
City^^'communi ty districts. One community district has already committed itself 
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Question ^iZ: What ty[)es of instructional prograiiis (or projects in 
the case of categoiical aid prt)granisl a re intjgt effective for students 
of different cha racte ris tics 

to implementing the LS.S, in all of its thirteen elementary schools for 
n\athematica m grades 1-6. This decision was made after several trips to 
the upstate pilot by the superintendent, central staff, principals, supervisors 
and program nianagers. In New York Cily meetings have been held with the 
community school boards, union representatives, teachers, parent groups 
and an advisory group established specifically for I S.S. 

This district realizes that a tren^endous amount of work is required at 
the outlet to organize the curriculum, develop the learning activities into 
tnodule.s (approximately two weeks of instruction), develop test items, and 
set acceptable levels of performance but feels that the payoff in terms of 
iterating sound programs for students once implemented by having feedback 
information is worth the effort. (Appendix A provides a description of activi- 
ties that make LE.S. an operational system. ) 

The I3epa rtment, through consultants, will j^rovice teclinical assistance 
to the participating districts in carrying out these activities and will adjust 
the computer support system to in^prove its efficiency. 

Additional community districts have been contacted and given detailed 
explanations regarding the LS.S. It ai>pcars tliat one additional district will 
become mvolved in^^h</'next fe<v ^m0ntlis. ' / 

& I ■ ' / 

:n upstate aAxl Lon^ Island districts, the CAM (Comprehensive ^chieve- 
ment Monitoring) software package is already being used in many districts. 
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Question Wluu ly\)v<y of ins ti'ucti onal programs (or projects in 

iho case (.)f c:at«j|.;oricaL aid prograras) are niost effective for students 

ol^ji ff 1? ren t rtui i'<i r jc s ti 

Efforts are underway to expand tlie coverage through B,O.C^.E.S. personnel, 

U\ addition, improvements have recently i)een made to the computer software 

package to increase its ability to trace learning activities, liandle biweekly 

mastery tests and lo generate additional reports to local personnel. 

In summ a ry , J_h )e | ni rini cnt is makin g substantial progress in providing 

lo£a 1 districts wi tli an ability to determine wl^iat instructional prog ra m s a re 

more succe ssful than cvthers for students of different characte r i s tics. Th e 

availability of such i nfo n nati on at the State [evel will clearl y enhance o u r 

capability t o dissen unale inforn i a tion to other school di st rit Is am[ t o the 

Governor, Leg islatu re a nd th^3 I^ oard of Regents . 

D . QUESTION r^ 3: TO WliA'I^ EX IGENT ARE THE BEST INS TR UCrriON AL 
PLANNING, MA.\7\C1EMEN T AND EVALUATION PRA(rnc:ES BEING 
USED AT IMP^ITyHEN r' LEVEIvS OF THE EDUCATIONAL SY ST EM? 

^' Inst r uc ii on ai Evalua tion System 

[he Department lias two major efforts underway to address this 
question. The first effort is c:onnected with the implen^entation of the Instruc- 
tional Evaluation System. As tlu; Instructional Evaluation System is imple- 
mented in schfjol fiistricts, nev. inethods of involving teachers and other 
district personnel in str\ictu ring and developing subject-matter curricula 
(i.e. instructional planning) are l:)eing explored. In addition, as districts are 
provided with previously imavailable information regarding the success qf 
various instructiotial approaches for cjiildren with different characteristics, 
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Question /^3; To what extent are the best instructional planning, management 
and evaluation practices bein^ used at different levels of the educational system? 

new roles and responsibilities are often required of teaching and supervisory 
personnel (i.e. instructional management and evaluation). Various techniques 
are being used. In essence, the Department's plan is to iterate toward im- 
proved instructional planning, management and evaluation practices as needs 
become known. 

2, Categorical Funding Study 

The second effort is one of systematically examining instructional 
planning, management and evaluation practices in a sample of school districts 
presently receiving funds for compensatory education programs (i. ESEA 
Title I and Urban Education). This study is called the Categorical Funding 
Study. Some preliminary findings of one part of that study are described 
below. The final report will be made in the fall of 1974. The Department 
believes that the recommendations in the final report will be generally appli- 
cable to all educational practices, not just to those connected with compensa- 
tory efforts, 

a. Purposes of the Study 

In order to more fully understand the preliminary findings 
reported below, all work activities underway in the Cate- 
gorical Funding Study, to be included in the fall report 
to the Governor and Legislature, must be outlined. The 
goal of the CategoricaT Funding Study is to recommend 
feasible, cost-effective means for the State Education 
Department to improve the planning, management, and 
evaluation of compensatory programs for the educationally 
disadvantaged. This goal arises from analyses suggesting 



Question ^3: lo what extent are the be^t instructional planning, tnanageinent 
nd ev aluation practices beinij used at differe nt Level s of t h e cducati ona l system? 

that relatively hi^h quality instructional planning, manage- 
ment, and evaluation processes will assist in achieving 
the desired outcomes of compensatory education. Tli^^re 
is a realiisation, however, that the in-\provement of these prac- 
tices is riot the only change needed to achieve better out- 
comes for students (other elements include the people 
involved, the educational approaches used and the like). 

The rationale for tlie study was threefold: 

First, categorically funded compensatory education 
programs have been attacked as costly, ill-managed, and 
ineffectual, leading to proposals for their elimination or 
radical restructuring. Because many proposals for changing 
aid programs assume a high quality of planning, manage- 
ment and evaluation skills and practices at the state and 
local levels the Department has concluded that this assump- 
tion should be tested in order to improve decisions in the 
future, 

vSecond, compensatory education is funded by a variety 
of grant mechanisms. Because much of the Department's 
programming involves grant programs, systematic study 
of the relative merits of alternative grant structures and 
grant management techniques would undergird many broad- 
ranging policy issues. This is especially in\portant at this 
time since at the State level there are under consideration 
proposals for applying a weighting factor to the State aid 
formula for educationally disadvantaged students. 

Third, improved understanding of existing local school 
district planning, management and evaluation processes is 
prerequisite to the design of comprehensive, in^proved 
systems for planning, management and evaluation. Because 
compensatory programs require more documentation than 
many other programs, study of them permits preliminary 
but systematic appraisal of local processes, 

i The Categorical funding Study consists of six major, 

related components. Below is a brief description of the 
objectives, procedures, status, and expected outcomes of 
each component: 
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Qviestion #3: To what extent are the best instructional planning, Jiianagemenfc 
and evaluation practices being used at different levels of the educational system? 

1) Study of the Situation . The purpose of this component 
of the study was to define the broad problems of com- 
pensatory education with sufficient precision to carry 
out the other components. This involved a series of 
library and original research efforts that culminated 

in the design of the balance of the Project. The results 
of these efforts are currently being documented. The 
direct benefits of this component will be review of the 
relevant literature and a clearer understanding of the 
problems in compensatory education. 

2) Functional Specifications for the Desired System. This 
component is intended to define a coherent set of speci- 
fications of the processes and structures that v/ ill con- 
tribute to the effectiveness of compensatory education. 

A synthesis of contemporary thinking regarding planning, 
management, evaluation, organization, and systems 
engineering is currently under development. The direct 
benefits of this component are definitions of and the 
relations between the most salient planninj^^, manage- 
ment and evaluation processes. These specifications 
will be useful in precisely defining detailed problems, 
in monitoring movement toward solutions, and tlieir 
application to areas beyond compensatory education. 

3) Study of Department Processes , This compojient includes 
documenting the processes and structures actually op- 
erating in compensatory programs in the Department. 
This study addresses the period 1969-1973 with a view 
toward appraising the impact of Department policies, 
guidelines and regulations, procedures and technical 
assistance on local district planning, management and 
evaluation practices and project performance. Inter- 
views and analysis of in-house documents, currently 
being conducted, are yielding the information necessary 
to complete this task. Direct benefits of this component 
will be a detailed longitudinal (i.e. multi-year^ descrip- 
tion of the Department's activities concerning compen- 
satory education. 
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Qiiestion H'i: To what extent are tlie best instructional planning^ inanagen>enl 
and evaluation practices being used at different levels of the educational system? 

4) Study of Local District Project Documentation , Phis 
component is intended to systematically measure various 
aspects of local district planning, management, and 
evaluation practices of compensatory projects on the 
basis of the applications and the evaluations that districts 
submit to the Department. This effort covers the period 
1969-73, as does the study of Department processes. 
Data collection and reduction are complet^id and some 

of the analyses of these data have been completed. The 
methods used and the preliminary analyses completed 
to date are described below. This component is pro- 
viding two direct benefits. The first is an empirical 
basis for more detailed understanding of local district 
planning, management and evaluation processes, the 
beginning of which is reported below. The second benefit 
is the development and testing of a series of instruments 
and procedures for measuring various cjualitative aspects 
of local district planning, management and evaluation 
processes that could be incorporated into ongoing 
Department project approval, monitoring, and evaluation 
procedures, i- '^thereby making them more objective and 
systematic. 

5) Department/ Local Level Modeling * This component will 
produce a preliminary descriptive/predictive model of 
the combined Department/ Local L>evel (within the frame- 
work provided by the functional specifications) from the 
results of the IDepartment descriptioii and the local level 
document study described above. This preliminary model 
will be "field verified" and refined through interviews in 
selected local school districts. The model will provide 
an assessment of the impact of Department actions on 
local performance, definition of the specific jn'oblems 

in the combined system, and means by whicli systt-matic, 
relatively precise, predictions of probable outcomes of 
various policy and program options can be made* 



Or those of any other State or F'ederal Agency 
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Question To what extent are the best instwictional planning, management 
and e v^ aluation prac ti ces bein^ used at different levels of the education a L sys tei n ? 



6) Trade-off Anal y ses . I'his final component will include 
alternative solutions to the problems identified in the 
previous section, and a comparison of these alterna- 
tives with respect to their feasibility and cost-effectiveness. 
This appraisal will provide the basis for recommending 
both policies and programs for the Department as well 
as workable strategies for local school districts and 
the Department to improve the planning, management, 
and evaluation of compensatory education at their 
respective levels. 

Within this context, the information to follow is an 
interim report of tlie findings of the fourth component. 
The next section describes the design, methods and pro- 
cedures used in the study of local level project documenta- 
tion. 

b. Procedures Used in the Study 

The objective of this part of the Project (i.e. the Study of 
Local District Project Documentation I was to examine the 
effects of different types of local school districts, programs, 
and time (independent variables)— on various aspects of local 
district planning, management and evaluation processes used 
in compensatory education projects (dependent variables). 
Three district types were considered: New York City Commu- 
nity School Districts (CSD^s), Large Upstate local school 
districts (i.e. districts outside New York City with more 
than 15, 000 students), and Moderate Ups tale Ibcal school 
districts (i.e. districts outside New York City having fewer 
than 15,000 students but receiving Urban Ed grants greater 
than $40,000 in 1971-72 ^'). 



As discussed later, additional independent variables will be used ir 
further analyses of these data. I'he effects of only these indeoen/ent 
variables are discussed in this interim report. 

The $40,000 grant size cut-off eliminated all districts with enrollments 
less than 5, 000 (small upstate) from the population. 
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Question ^3: To what extent ave the best instructional planning, n-kanagement 
and evaluation practices being used at different levels of the educa tional system? 

Three programs were included in the study, the State 
Urban Education program, ESEA Title I, and ESEA Title Hi, 
Urban Education and Title I both provide funds on the basis 
of pre-established formulas under approval of the Depart- 
ment, but differ in the source of the funds (State and Fede>-al, 
respectively), in the operative guidelines, and in certain 
aspects of Department processing. Title III is a grant 
program in which districts coziipete for funds on the basis 
of their applications. Comparison of the two formula grants 
permits appraisal of the effects of funding source, different 
guidelvnes, and Department administrative processes. 
Title III was used as an independent variable because its 
presence might indicate more aggressive local scliool 
district management {i. e. , success in competing for funds^. 

The time dimension for measurements taken from 
applications extends through the four school years, 1969-70 
to 197Z-73; but those from evaluation documents cover only 
three years, 1969-70 to 1971--72, because evaluation docu- 
ments for 1972-73 were not available when this activity 
was undertaken in the summer of 1973, 

Twenty-five districts containing a sample of 100 one- 
year E.S^E.A. Title I projects, LOO one -year Urban Educa- 
tion projects and 15 three-year E.S. E. A. Title III projects 
were selected for analysis in the study. Districts were 
selected based on the following criteria: (I) they operated 
both Urban Education and Title I programs and (2) their 
1971-72 Urban Education grants were greater than $40,000. 
The resulting group of 49 districts was tlien divided into 
those having Title III programs and those not having Title ill. 
Fifteen of seventeen districts having Title Hi projects were 
selected and ten of the remaining 32 districts were selected 
to give five districts in each of the following categories: 
(1) New York City Community School Districts (NYC/CSD^ 
having Title III projects; (2) NYC/CSD's not having Title ir 
projects; (3) moderate-sized upstate districts having T'itle I!!- 

(4) moderate-sized upstate districts not having Title and 

(5) large upstate districts (all of which also had Title 
projects). This 25-district sample was used ir the stuc.y. 
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Question M3: To what extent are the best instructional planning, management, 
and evaluation practices bein^ used at different levels of the educational systein? 

In selecting individual projects for each of these 
twenty-five districts, the following criteria were 
applied: the project was (1) continuous in terms of 
subject matter over the 1969-73 period, (2) funded at 
a level greater than $40,000, (3) cognitively oriented 
(generally reading or math), and (4) aimed at elementary 
grade students. If one of the criteria could not be met 
specific decision rules were followed for selection. 

An instrument was developed specifically for this study 
to measure the nature and quality of local schc ol district ^ 
instructional planning, management and evalua tion processes 
through application and evaluation documents. It was 
pilot tested in March, 1973, reviewed, discusr^ed and 
revised. 

The resultant instrument was composed of five groups 
of scales measuring planning, management and evaluation 
performance and one group of scales measuring responsive- 
ness to program guidelines. 

Eleven persons were hired for the task of rating the 
local school district documents. These project raters 
were given intensive eight-day training in the use of the 
instrument which involved a complete review of the instru- 
ment and scoring procedures and a two-day sample scoring 
exercise. 

The study instrument was applied to 215 project appli - 
cations and 159 project evaluations from three programs: 
ESEA Title I, Title III, and Urban Education. 

As indicated above, the instrument was composed of 
five groups of scales, three of which were included in this 
analysis; Application Quality (21 scales); Evaluation 
Qualiiv (15 scales): and Responsiveness to Guidelines 
(58 scales Title I, 22 and 13 scales for 1969-7 and 1971-73 
Urlan Education respectively); which were repeated a numbef 
of times for each district. 
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Question #3: To what extent are the hof^i instructional planning management, 
and evaluation practices being used at different levels of the e d ic ationa l system? 



To facilitate analysis and interpretation related scales 
were combined to form ^'indicators'^ of more complex be- 
haviors. Preliminary analysis included the f )llo\ving in- 
dicators constructed by grouping individual scales that 
relate to the behaviors under study: 

1) Application Quality (21 scales) 

2) Title I Responsiveness (58 scales) 

3) Urban Education Responsiveness, 
1969-71 (22 scales) 

4) Urban Education Responsiveness, 
1971-73 (13 scales) 

5) Evaluation Quality (15 scales) 

Because time did not permit a second pilot test of the 
instrument, a reliability substudy was conducted. Five 
scorers each scored the same three E.S. E.A, Title I 
projects and the Responsiveness sections u{ the same four 
Urban Education projects using standard scoring procedures. 
The major reliability issue was the extent to which different 
scorers used the scales in the same \yays. To assess this, 
inter -scorer reliability coefficients^'were computed for each 
study and are shown in Table 46. The coefficients are high 
enough to permit the conclusions that the indicators are 
reliable for the present purposes. 



The Pearson product-moment correlation coefficient (r) was calculated 
for each pair of scorers for each indicator yielding 10 correlation co- 
efficients each. These 10 coefficients were then "averaged*' by trans- 
forming each of them to Fisher^s coefficient, summing :hese scores, 



t 
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Question #3: To what extent are the best instructional planning^ management , 
and evaluation practices being used at different levels of the educational system? 

Table 46 

Reliability Coefficients of Indicators 

Indicator Reliability Coefficient 

I) Application Quality , 81 

Z) Title 1 Responsiveness . 55 

3) Urban Education Responsiveness, 1969-71 .73 

4) Urban Education Responsiveness, 1971-73 ,47 

5) Evaluation Quality .88 



The Application Quality scales deal with how well a 
district describes its proposed project in the application 
document submitted to the Department. The application 
quality indicator consists of 21 scales rating documents 
based on the following major groupings of criteria: 

(1) Needs - the precision with which the application 
describes district-wide and target group needs 
of educationally disadvantaged students • 

(2) Objectives - the measurability of objectives and 
the degree to which objectives are stated in student 
behavioral outcome terms. 



(3) Activities - the detail with which activities are 
specified. 

(4) Relationships - the degree to which objectives 
address needs and the degree to which activities 
address objectives. 

(5) Planning - the detail of description of the process 
by which the project was planned including the 
specificity of the consideration of alternatives, 

(6) Participants - the degree to which criteria used to 
select participants (professional staff, nonprofessional 
staff and students) are project relevant. 
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Question ^3: To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the ed u cational system? 

(7) Evaluation - a) the extent to which the planned 

evaluation addresses the stated project objectives; 
b) the extent to which issues of instrument validity 
and reliability are discussed; c) the precision of 
the evaluation design description; d) tlie relevance 
of comparisons proposed in the evaluation design. 

The Title I Responsiveness indicator consists of 58 
scales which are designed to measure a district's respon- 
siveness to official program guidelines. The scales vary 
from evidence of non-compliance to evidence of complete 
compliance with a selected sample of Title I guidelines ' 
(evidence of partial compliance is the midpoint on the scale). 
Title I program guidelines generally consist of criteria by 
which applications are considered for approval. The 58 
scales included in the Title I Responsiveness section, tlien, 
are based on selected groups of approval criteria* These 
groups include: 

/ 

(1) Selection of attendance areas; 

(2) Assessment of needs of disadvantaged children; 

(3) Planning and coordination of Title I program 
with other programs and agencies; 

(4) Program design considerations such as con- 
centration of funds on high priority needs, 
statement of program objectives and partici- 
pation of private school students; 

(5) Program implementation considerations such 
as selection and use of staff, staff training, 
parent involvement, use of funds for instruc- 
tion and facilities and dissemination of information; 

(6) Provisions for evaluating the program; 

(7) Comparability of services for disadvantaged children; 
and 

(8) New York State Title I priorities. 



Guidelines used in the Title I Responsiveness section of the instrument 
are principally based on Title I Program Guide #44 (INST. L203. i). 
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Question #3: To what extent are the best instructional planning^ management, 
and evaluation practices being used at different levels of the educational system? 

The Urban Education Responsiveness Indicator for 
1969-71 consists of 22 scales which attempt to measure 
a district's responsiveness to official Urban Education 
Progrcim guidelines in operation from 1969 to 1971. The 
Urban Education guidelines for 1969-71 generally prescribe 
the types of information v/hich should be included in an 
application. The 22 scales included in the Urban Education 
Responsiveness section for 1969-71, then, are based on 
the types of information prescribed in the guidelines such 
as : 

(1) Statement of objectives; 

(2) Description of target group(s ); 

(3) Description of the geographic area to be served; 

(4) Description of the proposed project including 
number and type of pupils, roles of personnel, 
number and type of staff, costs, etc. ; 

(5) Description of means for attaining coordination 
and achieving community participation; 

(6) Description of evaluation and dissemination 
procedures. 

The Urban Education Responsiveness Indicator for 
1971-73 consists of 13 scales which attempt to do the 
same thing as the 1969-71 scales using a revised set of 
guidelines. These revised guidelines also tend Lo pre- 
scribe, in certain instances, the types of information 
that should be included in an application (e.g. description 
of target groups). However, the revised guidelines include 
certain program limitation guidelines such as: 

(1) Limiting evaluation expenditures to 5%; 

(2) Limiting program administration expenditures 
to 10%; 

(3) Establishing minimum cost per participant 
at approximately $150; 
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Question #3; To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the educational system? 

(4) Establishing the ratio of teacher aides to 
participants at less than 1;20; 

(5) Determining that Urban Education funds should 
not be used to supplant federal, local or other 
state funds previously supporting a program. 

Evaluation quality scales deal with hov/ well a district 
describes its proposed project in the evaluation document 
submitted to the Department. The evaluation quality in- 
dicator consists of 15 scales rating documents on the 
following major groupings of criteria: 

(1) Objective . the measurability of objectives » the 
degree to which objectives are stated in student 
behavioral outcome terms and the extent to which 
the evaluation addresses the project objectives. 

(2) Evaluation Design - the precision with which the 
experimental design used in the evaluation is 
described and the appropriateness of comparisons 
proposed in the design as presented in the evalua- 
tion document. 

Activities - the detail of the description of project 
activities that were carried out. 



(4) Participants - the detail given on the characteristics 
of professional staff, paraprofessional staff and 
student participants in the project. 

(5) Data Analysis - the extent to which instruttient 
validity and reliability is described, the extent 

to which the evaluation document presents evidence 
of bias in the sampling of students to bo tested, 
and the degree to which data are summarizec' and 
subsetted. 

(6) Conclusions • the extent to which conclusions and 
recommendations are directly supported by data 
and are relevant for modifying the project in the 
future. 
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Question ff^: To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the educational system? 

c. Preliminary Results of Study 

The findings reported below related only to the application 
and evaluation documents , and not necessarily to the actual 
program operations in the field. 

The Application Quality indicator measures the extent 
to which local school district applications approach con- 
temporary management and planning standards. High 
quality applicatiojiS, those scoring near 1 on the indicator^ 
would closely approximate the best that might be expected; 
low scores, near 0, would indicate that the applications 
are far from planning theory norms. The applications 
being scored are the products of local school district 
planning efforts, and thus the Application Quality indicator 
may be interpreted as a ^'proxy" measure of planning 
quality. 

The mean Application Quality scores over all districts 
and years was as follows: Title I, . 51; Urban Education, 
. 48; and Title III, .51. A comparison of the three programs 
for all three district types (lb local school districts), found 
no significant differences in Application Quality scores 
among programs or district types (see Table 47), 



Table 47 

Mean Application Quality Scores 
by District and Program Type 



District Type 
Moderate Upstate 
Large Upstate 
NYC/GSD*s 



Title I 
. 5 98 
.480 
.490 



Urban Education 
.458 
. 538 
.492 



Title III 
.508 
. 474 
• 558 



v 
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Question ^3: To what extent are the best ins « .'uctional plannirfg> management, 
and evaluation practicevS being used at different levels of the educational system? 



A statistically significant trend of improving scores 
for both Title I and Urban Education projects over all 
local school district types from 1969-70 to 1971-72 (see 
Table 48) was noted, with a leveling off occurring in 
1972-73. This trend, however, onlvt represented a gain 
from a generally poor level of quality in 1969-70 to an 
^ overall fair level of quality in 1972-73, The NYC/CSD's 
showed the most marked improvement over the time 
period, starting at a lower level {. 349) but improving 
consistently to .565 over the time period. The moderate 
size upstate districts began at a higher level (, 434), 
improved less rapidly, and actually dropped very slightly 
in the last year to . 518 (see Figure 3). In addition, no 
significant overall differences were noted between the 
Urban Education and Title I programs over the time period. 
However, for moderate upstate districts. Title I appli- 
cations had higher scores than Urban Education applica- 
tions, while the opposite was true for large upstate 
districts. No difference was observed between programs 
for the NYC/CSD's (see Figure 4). 



Table 48 

Combined Title I and Urban Education 
Application Quality Scores Over Time 





1969-70 


1970-71 


1971-72 


1972-73 


Combined Title I and 










Urban Education scores 










for all district types 


.407 


. 483 


. 544 


, 551 



Overall, despite improvements in some district types, 
application quality scores were relatively low over the 
four -year period under study when compared to contempo- 
rary planning standards. Further interpretations of the 
significant differences noted above will only be made after 
further analysis of the data. 
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Question #3: To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the educational system? 

The Responsiveness indicator measures the extent to 
which project applications reveal conformance to the guide- 
lines and regulations operative in the program and year in 
which the applications were submitted. The application 
themselves (not necessarily the operating projects) must 
describe behavior in conformance with the regulations. 
Explicit mention of the guideline or descriptions of con- 
forming behavior in the documents is interpreted as evi- 
dence that the districts have acknowledged the guideline 
in planning the project. This interpretation is based on 
the fact that these documents are the principal source of 
information by which the Department project approval 
process verifies local school district conformity with 
guidelines. The guidelines and regulations differ between 
the programs; the Urban Education guidelines were revised 
during the period of this study so separate indicators were 
used for Title I, 1969-71 Urban Education, and 1971-73 
Urban Education projects. 

The mean combined responsiveness score for all 
districts and years was .644, indicating a moderately 
high degree of overall conformance to guidelines and 
regulations. 

When considered over time the combined responsive- 
ness scores for the New York Community School Districts 
and moderate size upstate local school districts were about 
the same in 1969-70 and 1970-71. During the last two years/ 
of the study period, however, the New York city district's 
Responsiveness Scores improved, while the moderate up- 
state district's responsiveness scores dropped. 

When the Title I and Urban Education programs were 
examined separately, the following mean scores were re- 
ported: Title I Responsiveness for all districts and over 
all years . 59; Urban Education Responsiveness for all 
districts from 1969-71 . 68; Urban Education Responsive^ 
ness for all districts from 1971-73 . 69. The above means 
represent statistically significant differences in the two 
programs with Urban Education scores generally higher 
over the four-year period than Title I scores. Although 
the scores of the two programs were significantly different 
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Question ^3: To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the educational system? 

statistically, it must be noted that substantial differences 
exist in the sets of guidelines^ and therefore, scales used 
to measure responsiveness to those guidelines, result In 
different scores. Title I guidelines tend to. emphasize 
program details and criteria by which applications will be 
considered for approval. By contrast, the Urban Education 
guidelines are more oriented to procedural issues (e. g. how 
to format applications, what information to include in appli- 
cations, etc, ). Consequently, districts which score high 
in reg|ird to the guidelines of one program may not neces- 
sarily score high in relation to the guidelines of the other 
program. Analysis involving program and district type 
revealed significant findings consis tent v;ith the above 
hypothesis. For example, moderate upstate districts had 
about the same level of responsiveness^ on both Title I and 
Urban Education over the four-year period. However, the 
New York CSD's scored lower on Title I than on Urban Edu- 
cation, Additional comparisons between programs must 
await further analysis. 

Overall, responsiveness indicators showed a moderately 
high degree of conformity between applications submitted by 
local school districts and the operative program guidelines 
in the two programs under study. 

The Evaluation Quality indicator measures the extent to 
which the evaluation documents aubnntted by districts to the 
Department approach contemporary project evaluation stand- 
ards. High quality evaluations (scores approaching 1.0) 
would clearly approximate the best that might be expected; 
low scores (near zero) would indicate very poor evaluation 
performance* Since the evaluations scored are either the 
products of 4istJ^icts' own efforts or the products of contract- 
ors working under the supervision of local officials, the 
Evaluation Quality indicator may be interpreted as a "proxy" 
measure of the quality of districts' evaluation processes 
or capabilities. 

The mean Evaluation Quality indicators for all local 
districts and time periods were * 44 for Title l and . 48 for 
Urban Educationi indicating a rather poor quality of evalua* 
tion documents* A significant upward trend in combined 
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Question #3: To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the educational system? 

Evaluation Quality scores from ,416 to ,492 was found over 
the three years for which evaluation documents were scored. 
However, this overall improvement can be principally attri- 
buted to the increase in New York CSD's which had an ex- 
tremely low mean score of • 32 in 1969-70 and moved to a 
fair level of • 61 in 1970-71, Both the large and moderate 
upstate districts remained relatively the same, the means 
ranging closely around .45 for all three years. Table 49 
and Figure 5 represent the combined Evaluation Quality 
Scores for the three district types over the study period. 

Table 49 

Combined Title I and Urban Education Evaluation 
Quality Scores by District Type Over Time 





1969-70 


1970-71 


1971-72 


Moderate Upstate 


.456 


.452 


.467 


Large Upstate 


.470 


. 432 


. 474 


NYC/CSD'8 


. 321 


.514 


. 534 



In summary, Evaluation Quality was found to be fairly 
stable at a relatively low level over the three*year period. 
Title I and Urban Education projects scored about the same 
on this variable. Significant improvement over the three- 
year period was noted for New York CSD's. 
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Question #3j To what extent are the best instructional planning, management, 
and evaluation practices being used at different levels of the educational system? 

* 

Limitations of these Data 

As indicated above only preliminary analysis of these data 
has been performed* Further analysis is currently under- 
way / Ali^in^^ 

through field visits to local districts . An intensive, 
structured interview is planned to determine: (1) the planning 
and evaluation processes as they exist at the local level and 
(2) the correspondence between those processes and the 
processes reported in the Title I and Urban Education appli- 
cation and evaluation documents submitted to the State 
Education Department. 

e. Summary of Findings 

Responsiveness scores were relatively high in both the 
Urban Education and Title I programs, in all local school 
district types and in all study years. General upward 
trends in scores over the four-year period were noted for 
Application Quality scores for all district types in the two 
pro grams. Similar improvement was found over the study 
time period in the Evaluation Quality scores for New York 
City community districts in both programs. Evaluation 
Quality sc ores for large and moderate upstate districts 
for both programs remained relatively stable over the 
three -y ear period. Despite existence of these improve- 
ments in scores, both Application and Evaluation Quality 
scores were still found to be quite low when compared to 
c o n te mp o r a r y planni ng and e va lua ti o n s tanda r d s . 

These findings represent only a preliminary analysis 
* of the data. Intensive field interviews will be conducted 

in order to describe actual local school district operations 
more precisely* Further analysis and comparisons of 
data obtained from both the scoring of documents and the 
interviews will be performed prior to making recommenda- 
tions in fall 1974. 
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A. Spring 1974 - Workshops 

1) Formulate or revise program (e, g» reading) and subprogram 
{e.g. grade 1) objectives 

2) Divide subprograms into modules 

3) Form banks of instructional activities (employing specified 
or new, teacher-generated materials) linked to objectives 

Activities 

Three types 

(1) '^First^try'' educational processes 

(2) "Compensatory" processes 

(3) "Enrichment" processes 

4) Generate mastery test items to measure the extent to which 
instructional objectives have been met by students 

5) Determine the particular type and extent of individualized 
instruction to be employed 

6) Determine the organizational means of implementing instruction, 
e*g. self-contained classrooms, team teaching, etc. 

7) Set mastery standards, i.e., what proportion of test items 
associated with each objective must be correctly answered to 
establish mastery of that objective. 

8) Decide on roles and missions of all personnel connected with 
the program. 

For examples 

*-From whom can a teacher seek help when it is needed? 

>«-Who is responsible for takin action if students do not meet 
district standards? 

B. Summer 1974 - Preparation for Implementation 

1) Before the school year begins, mastery and retention tests for 
all modules must be prepared and Inventoried 
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2) Machine -readable mastery test answer sheets or forms must 
be produced 

3) Program manuals describing modules, objectives, instructional 
activities (employing designated materials), and performance 
criteria for each objective must be made up and distributed 

Ci Beginning of the School Year 1974--75 

1) Students are assigned to teachers 

2) Elements of pupil, personnel and program files necessary to 
implement ISS are set up 

3) Individually addressed placement tests (with student^s name, 
I, D, number, school and teacher's name pre^entered) are 
given to students to enable teachers to determine which 
objectives will be taught to different students or groups of 
students 

4) Test request forms listing all students assigned to each 
teacher are prepared and delivered to each teacher together 
with all other ISS materials 

During the School Year 1974-75 

1) Instructional activities and resources are selected from the 
banks by teacher (with or without student participation) and 
delivered to individual students or to groups of students by 
teachers 

2) As each student or group of students n'ears the end of a module, 
the teacher files a teat request form, indicating the required 
test number and checking off the students to be tested 

3) In a few days, individually addressed mastery tests (with 
answer sheets) are delivered to the teacher 

4) The teacher administers the tests and returns the answer sheets 
for scoring 

5) Within twenty-four hours^ test scores (by objective) are reported 
to the teacher and to the students 
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6) The teacher modifies instruction {e.g., selects a set of activities) 
basedA>n a diagnosis of test score results 

7) After some time, retention tests for each module are requested, 
administered, scored, and reported 

8) All records of test scores and instructional activities adminis- 
tered are kept in computerized files for analysis and evaluation 
of module and subprogram effectiveness, 

9) Monthly summary and exception reports of student performance 
are sent to administrators. Other reports can be generated on 
request 

E. After the School Year is Over 

1) An evaluation of overall district performance by program is 
executed, and a report having a form and content specified by 
the district is generated 

2) Analyses of pupil and program files are made to identify 
ineffective activities as a function of learner characteristics 

3) Analyses of the differential effectiveness of activities by learner 
characteristics are executed and a decision is made to enter the 
results in descriptive , language into the cM.Urict^a I3S program 
books (e. g» , for a particular activity, tlie Qutry might be '^works 
particularly well for students who speak non-standard English^') 

4) Analyses to support budget-making and resource allocation 
decisions are executed as required 
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I. INTRODUCTION AND PURPOSE STATEMENT 



INTRODUCTION 

Periodic, rigorous polling of the various populations com- 
prtsing the ''public*' concerning their perceptions of general 
trends and critical issues In public education Is crucial to retain- 
ing public support and confidence for school programs. This 
pulse-taking, to be effective, needs to occur on national, stale, 
and local levels, utilizing a variety of sampling techniques. Al- 
though systematic collection of information relative to public 
attitudes and preferences is common In many areas of govern- 
mental service, it has been— unfortunately — a rare activity In 
public education. 

A notable exception is an annual survey of public attitudes 
toward education conducted by the prestigious Gallup polling 
organization since 1969, results of which are printed each fall In 
Phi Delta Kappan, This attitude measure is obtained by interview 
techniques from a rigorously designed ''modified probability 
sample" of over 1500 adults. This sampling is a truly representa- 
tive microcosm of the entire nation, having been selected using 
the latest available census figures. In addition to the obvious 
function of providing precise current information on the attitudes 
of the general public, the Gallup data also furnish a reliable 
indicator of developing trends and a bench-mark against which 
local and regional studies can be compared. 

Each year several specific issues have been selected for 
Gallup Investigation. The format used by the Interviewers also 
Includes several more general, permanent questions, foremost of 
which Is the critical Issues query: What are the biggec^t problems 
for the pub(ic schools fn your community?** Issues most frequently 
cited in the past four years are summarized in Table 1. 

The top three issues cited In each of these Gallup surveys: 
discipline, finances, and integration, were consistently high se- 
lections, Indicating strong, on-going public concern. Drugs and 
the quality of teachers were other concerns which consistently 
received high ranking. Concern about the adequacy of school 
facifitfes has declined as a significant issue, ^/hile the pupil- 
teacher ratio has appeared In the listing as a matter of critical 
import to the adult public. Gallup has noted that the top three 
issues are ones that easily tend themselves to front-page news- 
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paper coverage. He also points out that there [s growing senti- 
ment on the part of the general public to call for accountability 
and improved efficiency from the public schools (1973). 

In addition to this primary survey, Gallup annually asks the 
same questions of a smaller non-representative sample from a 
specific population sub-group. High school students were polled 
in 1970 and 71 and a mixed group of educators was queried in 
1972 and 73. Students differed from the national sample of adults 
in thai they showed considerably less concern about school 
financing, but saw more problems in the areas of teacher com- 
petence, inadequate facilities, and poor curriculum. The opinions 
of the educators regarding critical issues closely resembled 
those of the general public, with several notable exceptions. The 
educators perceived school financing as the uncontested primary 
problem, with parent-student apathy, curriculum, and facilities as 
other issues of high concern. Educators were less concerned 
about teacher competence than was the general public. 

The Gallup effort has inspired several other studies on na- 
tional or regional levels, usually involving a sample of some 
specific sub-group rather than the general public, and usually 
focusing on specific single issues or potential problem areas 
rather than a full measure of all critical issues. These studies are 
therefore simitar to the Gallup sub-group effort and to the chang- 
ing latter portion of the main Gallup surveys, but do not provide 
results that are comparable to the primary data collected by 
Gallup. Also the Gallup organization's precision and objectivity 
often are lost In the replications. 

Wills and others (1972) used the Gallup critical issues query 
in sampling 496 secondary education majors at two major univer- 
sities. This specialized population responded by listing an aver- 
age of more than three issues per respondent (vs. IV4 obtained 
by Gallup from the general public). These teachers-to-be did not 
cite integration and teacher competence, two of Gatlup's con- 
sistent leaders, as significant problems in education. They did, 
however, show high concern with drugs, finances, curriculum, ^ 
parent -pupil apathy, and administrative and school board policy. 

Savage and Jones (1972) chose to focus on opinion of par- 
ents, using a questionnaire which was distributed randomly to 
180 parents In 9 scattered states. This sample was not represen- 
tative of the general public^ but focused on suburban, middle- 
class families. The instrument was structured — not open-ended 
—and was not made available for critical scrutiny, leaving uncon- 
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firmed the suspicion that the opinions obtained may have been a 
function of the nature of the instrument. Savage and Jones found 
their suburban parent sample concerned most about the quality 
of instructional practices and personnel. Parents were generally 
satisfied with the school system, suspicious of educational ex- 
perimentation or innovation, and showed tow levels of concern 
about drugs, integration, and school finance. Parent responses 
tended to be very specific, closely related to the educational 
experience of their own youngsters, and seemed to show special 
sensitivity for the impact of various educational practices (student 
evaluation, grouping, testing) upon the self-concept of the child. 
The opinion of the Savage and Jones sample of parents appeared 
to resemble closely that found by Gallup among educators and 
by Wills among teachers-to-be, but was sharply divergent from 
the concerns of the general public, as tapped by the primary 
Gallup data. 

In 1972 The National Education Association conducted a 
thorough study of instructional problems as perceived by a 
representative sample of almost 2,000 public school teachers. 
Although this study did focus on a specialized area (instruction), 
rather than measuring fully the broad range of critical issues, it 
is of some value in indicating the nature and degree of teacher 
concerns in that area. The four instruction-related problems most 
cited were: (1) pupil apathy, (2) over-crowded classes, (3) burden- 
some non-instructional duties, and (4) heterogeneous pupil 
grouping. 

Of interest to Tennesseans is a regional needs assessment 
undertaken in 1971 by the Appalachia Educational Laboratory 
(Campbell, 1971). This study was designed to pinpoint critical 
educational problems in the region and select organizational 
objectives for 1976. Results from two surveys of school personnel, 
a survey of regional '*experts'^ and a conference of 200 **decision- 
makers" were utilized in making these determinations. The result- 
ing concerns and objectives focused on (1) improving reading 
skills, (2) emphasis on vocational and career education, and (3) 
efficient educational leadership and organization. 

Thorough survey of related literature on the State level dis- 
closed no recent, systematic study of critical issues in education 
on a state-wide basis in Tennessee. Even marginally related 
studies tended to be outdated, partisan, or haphazard. 

The Tennessee Education Association (1974), in developing 
Its proposed legislative action program for 1975-76, attempted to 



comprehensively survey teacher concerns. Opinion was collected 
from a "tear-out" survey form published In the journal Tennessee 
Teacher and from minutes of ten regional meetings caKed spe- 
cifically for the purpose of collecting such information. Although 
the tabulated results were not presented as statistically repre- 
sentative of all teacherS; and tended to focus on specific pro- 
fessional concerns which could be legislated (e.g,, duty-free 
lunch privileges, methods for calculating attendence, sick leave 
policies, retirement) rather than overall critical Issues, they did 
give an informal barometer of teacher concerns. The responding 
teachers showed high levels of concern in three broad areas: 
(1) school financing, and staff salaries and fringe benefits; (2) 
Increased staffing resulting in lower pupil/teacher ratios and more 
assistance from specialized teachers, and (3) professional Im- 
provement through rigorous standards and general teacher-train- 
ing overhaul. The teachers did not indicate concern regarding 
Integration, discipline^ pupil-parent-public apathy, or school facil- 
ities. 

Responding to growing public criticism of State-supported 
education, the House of Representatives of the 88th Tennessee 
General Assembly established a select study committee which 
conducted twelve days of hearings in seven cUies during the fall 
of 1973, Testimony was collected from individuals representing 
various groups with high interest in, or strong opinions regarding, 
public education. The resulting staff report cited three majcr areas 
of concern: pupil deficiencies in basic skills, school discipline, 
and reduced public confidence in the educational system. Seven- 
teen recommendations were addressed to ameliorating these 
concerns, most focusing on intensification of reading instruction 
in primary grades, improvements in teacher training, and admin- 
istrative expansion and reorganization. The establishment of this 
Committee was a strong indication of government awareness of 
public discontent with the schools, and of the desire for accurate 
information on current opinion regarding critical issues. How- 
ever, public heari^ngs, vulnerable as they are to the pressures of 
special interest groups, do not provide a highly valid means for 
coltectiiig the objective information desired. The staff report 
reflected the inadequacies of both the data collected and the 
supporting staff. 

Several deficiencies seem obvious in analyzing efforts to 
collect opinion on critical issues in education. 



ERIC 



(1) There is a need for careful, systematic studies of attitudes 
and preferences of the general public on local and state 
Iwels, 

(2) There is a definite need for comprehensive comparative 
studies which use the same instruments and procedures 
to poll the numerous sub-groups of educators, govern- 
mental officials, parents, and general public. Comparisons 
between dissimilar studies of the specific sub-groups* 
opinions (regarding critical issues) are of limited validity 
and usefulness. 

(3) Local studies, to be of real value and high reliability, re- 
quire careful design, implementation of the design, and 
interpretation of the data. The slip-shod character of 
many locaf and slate data collection and interpretation 

, efforts is a disservice to the public, and provides a target 
for ridicule of educational research by the scientific com- 
munity. 

The present study was undertaken with the intention of con- 
centrating on three areas of critical deficiency. The study en- 
compasses: 

(1) a full, open-ended study to provide a broad measure of 
critical issues in Tennessee education (Phase One) 

(2) a systematic, replicable, and objective measure of critical 
issues opinion (Phase Two) 

(3) a comparative measure of opinion of six distinct sub- 
groups. 



PURPOSE 

The purpose of this study was to obtain objective evidence of 
the opinions of Tennesseans concerning the critical issues in 
education in the State during the period 1973-74. This involved 
(1) identifying no more than ten key educational issues, then (2) 
having the issues ranked in order of importance by six groups 
of Tennesseans involved directly or indirectly in the educational 
process. 
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II. SURVEY DESIGN AND IMPLEMENTATION 



PHASE ONE 

To facilitate the gathering of objective data from a large sam- 
ple of Tennesseans, a concise listing of current eductlonal con- 
cerns was needed. The method chosen for identifying the issues 
for this listing involved mailing a preliminary survey Instrument 
to a smafi sample of the same groups that later participated in 
ranl<ing the Issues. 

Six groups of Tennesseans were selected to represent pro- 
fessional and lay opinion with regard to education Issues In the 
State. Superintendents, principals, and teachers in public school 
systems were surveyed to obtain the opinions of professionals. 
As representatives of the public exercising fiscal and/or policy- 
making responsibility in connection with public education, quar- 
terly county court members, city council members, and school 
board members were chosen to provide the lay point of view. 

The preliminary survey Instrument consisted of a 5" x 8" 
business reply card on which the respondent was asked to *'list 
as many as you wish of what you consider to be the most critical 
Issues, or pressing concerns, In education In Tennessee today." 
This card, accompanied by a letter explaining the purpose of the 
survey (both card and letter are included in Appendix A), was 
mailed to at least four individuals in each of the State*s 146 
school systems: one teacher, one principal, one school board 
member, and one city council or county court member (whichever 
group appropriated funds for the system). Fifteen (or 10% of the 
146) superintendents also received the preliminary Instrument, 
so fifteen of the State's systems were represented in the Initial 
survey sample by five individuals. Every Individual selected to 
participate in the initial survey to identify critical issues was 
chosen at random from the current list of the members of his 
particular group in his school system (e.g., the school board 
member selected to represent the Knox County system was 
chosen at random from a list of Knox County School Board 
members). 

The list of Tennessee teachers from which selections were 
made came from the State's PR-2 computer tape obtained from 
Dr. Howell Todd, Coordinator, Planning and Evaluation, In the 
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Stafe Department of Education. The list of principafs and super- 
intendents was taken from the State Department's Directory of 
Public Schools for 1972-1973. 

Names and addresses of school board members were ob- 
tained from Tennessee School Boards Association Directory of 
Superintendents and School Board Members, State of Tennessee. 
January 15, 1973. Names and addresses of city council members 
associated with each city, town, or special district school system 
were taken from Directory of Tennessee Municipal Officials. 
1973-74. (Eckard, 1973) compiled by the fvluniclpal Technical 
Advisory Service Institute for Public Service at the University of 
Tennessee, Knoxville. Finally, the Tennessee County Services 
Association, Nashville, provided a list of the names and addresses 
of quarterly county court members in the State. 

Fewer than 200 of the individuals who received the prelimi- 
nary survey instrument, which was mailed early in October 1973, 
took time to write down their ideas aboUt current critical issues 
and return the business reply card. Although the response rate 
was low, the list of Issues mentioned most frequently by this 
sample of Tennesseans bore a striking resemblance to the list 
compiled by Gallup in his national survey of public attitudes 
toward education In 1973. Thus it was felt that the results of the 
probe with the preliminary survey Instrument could provide a 
valid starting point for the concise listing of current educational 
concerns needed for Phase Two of the survey. 

Since Phase Two sun^ey participants would be asked to rank 
a set of educational Issues In order of importance, the decision 
was made to construct a listing of issues that would include no 
more than seven to ten of the most frequently mentioned Issues 
from the preliminary survey instrument. As it happened, the 
frequency-of-mention criterion rt'^suited in the Identification of 
ten issues which were clearly o' more concern to Phase One 
respondents than the other items they mentioned. 

The Issues 

A considerable amount of effort was expended to name the 
\ssues in as concise a manner as possible for listing, with In- 
structions, on a second 5" x 8" reply card. The Issues were 
purposely stated in a general way so as to encompass both pro 
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and con positions that might exist in the population to be sur- 
veyed. 

The list of issues (not in order of Importance) which finally 
was obtained from responses to the preliminary survey instru- 
ment included: 

Teacher competence Financing education - Includ- 

Vocatfonal education ing salaries 

programs Discipline 
Size of classes-overcrowding Lack of concern by pupils, 

and overloaded staff staff, parents, and public 

Improvement of general Inadequate facilities 

curriculum Administrative reform and/or 

Special education programs reorganization 

The term teacher competence was used to encompass a 
range of concerns related to the abiiity of teachers to carry out 
their responsibiiities in a satisfactory manner. Respondents ex- 
pressed feelings that some teachers lacked maturity, a sense of 
responsibility, a professional attitude toward their work. Some 
teachers seamed to have a poor attitude toward studer;ts or were 
too permissive, or perhaps too militant, according to the re- 
sponses received. Involvement of politics in the hiring and firing 
of teachers was not seen as a positive contribution to the im- 
provement of teacher competence. Some respondents were con- 
cerned about tenure policies, feeling that older teachers were 
not sufficientiy adaptable to change and that the granting of 
tenure made il difficult to replace Incompetent personnel. Per- 
formance evaluation was mentioned as a crjuclal need In the 
endeavor lo improve teacher competence; more realistic pre- 
servlce and in-service education, and up-grading of teacher 
preparatory programs at Negro colleges and universities were 
also mentioned in this connection. Due to shortages of trained 
professionals in such areas as special education, pre-school 
education, mathematics, art, and music, some teachers were 
teaching subjects for which they were not certified or even 
trained. Other teacher weaknesses causing respondents concern 
included poor communication skills, Ignorance of behavioral 
modification techniques, Inability to utilize paraprofessional as- 
sistance efficiently, and lack of competence in using audio-visual 
equipment or materials. 

The national emphasis given to career education in 1973 
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doubtless was responsible for the strong interest on the part of 
survey respondents in vocational education programs. A state- 
wide program of career education aimed at all grade levels with 
sufficient funding for adequate facilities and well-trained per- 
sonnel was the goal expressed. 

Teachers, principals, and superintendents responding to the 
request to identify critical issues expressed concern about size of 
classes. Most seemed to feel that failure to enforce Slate guide- 
lines for class size was the chief problem in this area. With more 
students in a classroom than the room was designed to hold, 
and/or more students in a class than a single teacher should be 
expected to teach, individualization of instruction is impossible, 
and any kind of instructional program is thereby jeopardized. 

Vocational and special education were specific curricular 
areas that seemed to be of particular interest to survey respond- 
ents in 1973 74. But improvement of the general curriculum also 
ranked high as an area of concern, especially among city council, 
couniy court, and school board members. Individuals represent- 
ing these groups called for more emphasis on reading, writing, 
and arithmetic, with special attention given to reading. "Get back 
to basics," seemed to be their message. There was disagreement, 
however, with those who fell that the current proliferation of 
courses was responsible for a decline in the quality of basic 
learnings. Some respondents felt that, especially In small schools 
and in rural areas, more olectives should be offered In order to 
Increase student Interest and motivation. Suggested additions to 
the curriculum included art, music, science, mathematics, kinder- 
garten, office education, physical education, and foreign lan- 
guages. Periodic evaluation and consequent revision of the 
curriculum received strong recommendations. 

Like vocational education, special education was in the spot- 
light both at the national level and at the state level in 1973-74. 
Survey respondents expressed concern about educational pro- 
grams for the gifted as well as the physically and mentally handi- 
capped. Needs for more money, better facilities, and more trained 
personnel were indicated. Teachers and administrators also 
voiced serious concern about the results of placing educable 
mentally retarded students in regular classrooms (a practice 
which may be necessary in some situations due to recent State 
legislation requiring that all handicapped students be given 
access to public schooling). 
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Some survey respondents expressed the opinion that financ- 
ing education was the only critical issue in Tennessee education: 
with sufficient funds all the other concerns could be alleviated. 
The initial survey instrument brought in more references to 
money and money-related matters than to any other issue. Teach- 
ers wanted higher salaries; administrators wanted more money 
for facility maintenance and improvement; and the lay groups 
wanted the State to furnish a larger percentage of the funds than 
ever before. Some respondents were ready to join a crusade to 
replace Tennessee's property tax with an income tax In order to 
Increase the total of funds available for education. 

Discipline was the term used by some respondents to refer to 
the problems with students th^t had resulted from a deterioration 
of their respect for authority. Both verbal and physical student 
attacks on teachers were cited as evidence of this deterioration. 

Discipline and lack ol concern are related issues since un- 
doubtedly discipline problems result, at least in part, from a lack 
of concern on the part of pupils, staff, parents, and the public. 
Respondents felt that community support of schools was at an 
all-time low, the public having lost faith in tho educational process 
employed in the public schools. Parents were not Interested in 
becoming involved in the education of thoir children nor in en- 
couraging their offspring'to do well in school. Refusal of parents 
to support the school in disciplinary matters was mentioned as a 
further indication of their lack of concern. Some teachers were 
responding to the indifference of others with apathy of their own. 
Lack of dedication on the part of some teachers was cited as a 
critical educational issue by several teachers and administrators. 

Many survey respondents were troubled by the existence of 
inadequate facilities: outdated physical plants, poorly maintained, 
housing more students than designed for, with dim prospects for 
obtaining new facilities or needed Improvements in existing ones. 
Facilities for libraries, indoor play, and vocational courses headed 
the list of needs mentioned by respondents. 

Administrative reform and/or reorganization appeared to be 
the required solution for a variety of concerns about operations at 
both local and State levels. Confusion and instability in the State 
Department of Education, as well as lack of innovative leadership 
there, were pointed out by several respondents. Some suggested 
a State-wide reduction in the number of teachers a school must 
have before a full-time principal could be appointed. The need 
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for more effective channels of communication between state and 
local education agencies was mentioned. Personnel working in 
small schooi systems suggested combining several small sys- 
tems, perhaps even going so far as to specify that no county 
have more than one school system. At the local level the following 
administrative changes were suggested: Improving money man- 
agement and cutting waste in school budgets; increasing teacher, 
pupil, and parent participation in school administration; changing 
the method of naming a school superintendent (from election to 
appointment by the board of education or vice versa); opening 
more administrative positions to blacks and other minorities; re- 
moving politics from the hiring and firing of teachers; changing 
the school calendar to permit twelve months of operation; and 
investing fiscal responsibility for the schools in the board of 
education. 

PHASE TWO 

Responses to the preliminary survey instrument resulted in a 
list of ten most-frequenlly-mentioned educational issues. This 
list, along with instructions for evaluating the issues in two ways, 
was printed on a 5" x 8" business reply card. The card and an 
explanatory letter (see Appendix A for a copy of each) were 
mailed to ail superintendents and to a sample of teachers, prin- 
cipals, school board members, county court members, and city 
council members throughout the State. 

Directions to Respondents 

Survey respondents were asked to Indicate In two ways the 
relative importance of the ten education issues listed on the reply 
card. Instructions to rank the issues in order of importance from 
1 (most critical issue) to 10 (least critical issue) forced the re- 
spondent to attach some significance to every Issue. A given 
respondent might have considered only two or three of the issues 
to be of key importance, but he had to rank all of them. Thus 
issues given rankings in the middle range by this individual would 
actually be receiving more weight than he felt they should have. 

Consequently, a second scale was added to the form to allow 
the respondent to rate the issues 'A' (of critical importance), 'B* 
(of some importance), or (of little or no importance). Thus the 
individual who considered only three Issues to be of real Im- 
portance coufd rate those issues 'A' and all others 'C. A respond- 
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ent who felt all ten issues were worthy of serious consideration 
could rate all ten 

ParaHel Forms 

To minimize the possibility that the order of presentation of 
the Issues on the reply card would have an effect on the order of 
the rankings^ two forms of the reply card were printed. On a 
yellow card the list of issues began with 'teacher competence' 
and ended with 'administrative reform'. On a blue card the listing 
was reversed: 'administrative reform' appeared first and 'teacher 
competence' lasL The two forms were alternated so that within 
each of the six groups surveyed one-half the sample was sent a 
yellow form and one-half was sent a blue form. 

The Sample 

Some local school boards and city councils in the State con- 
sisted of only two members. Thus, to give each of the 146 school 
systems an equal opportunity' for representation from lay policy- 
making bodies and fiscal authorities, the Phase Two survey 
Instrument was mailed to two school board members from each 
school system; and to two city council members if a city, town, 
or special district system, or to two quarterly county court mem- 
bers if a county system. (Where fiscal authority for a county 
school system was exercised by a county council or metropolitan 
council, two members of this body were included in the sample 
In the place of quarterly county court members.) 

In order to obtain the broadest range of opinion from these 
politically sensitive groups, r^o school board, city council, or 
county court member who had been contacted for participation 
in the Phase One survey was included in the Phase Two sample 
unless the body consisted of only two members. In order to select 
the sample from each governing body the members of the school 
board, and of the city council or county court associated with 
each system, were assigned numbers and a table of random num- 
bers was used to determine which individuals would be included 
in the sample. 



^ Since there was no consistent felallonshlp between size of school system 
and size of school board, city counci^ or county court, there was no point In 
an&mpYwg to devise a proportional method of representation based on size of 
school system. 
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For Iwo reasons the entire population of school superintend- 
ents was included in the Phase Two survey; (1) this was an im- 
portant group and small enough (146) to include without sampling, 
and (2) in order to compare the responses of elected superin- 
tendents with those of appointed superintendents, it was desirable 
to maximize the total of respondents in the superintendent cate- 
gory. 

To facilitate selection of a stratified random sample of prin- 
cipals and teachers, the schools included in Directory of Public 
Schools for 1972-73 (Tennessee State Department of Education, 
1973) were listed according to a six-fold classification scheme: 



County System 


City, TowDi Special 
District System 


(1) Elementary School 

(2) Secondary School 

(3) Combined School (Grades 

0-12 or 1-12) 


(4) Elementary School 

(5) Secondary School 

(6) Combined School (Grades 

0-12 or M2) 



A school was classified as an elementary school if, according to 
the Directory, the grade spread was; 



0-1 


1-1 


2-2 


3-3 


4-4 


5-5 


6-6 


7-7 0-8 


0-2 


1-2 


2-3 


3-4 


4-5 


5-6 


6-7 


7-8 


0-3 


1-3 


2-4 


3-5 


4-6 


5-7 


6-8 


7-9 


0-4 


1-4 


2-5 


3-6 


4-7 


5-8 






0-5 


1-5 


2-6 


3-7 


4-8 








0-6 


1-6 


2-7 


3-8 










0-7 


1-7 


2-8 












0-8 


1-8 















A school was classified as a secondary school if its grad« spread 
was given as: 



7-10 


8-9 


9-10 


10-10 


11-11 


12-12 


7-11 


8-10 


9-11 


10-11 


11-12 




7-12 


8-11 
8-12 


9-12 


10-12 
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This classification scheme yielded the following numbers of 
schools in each category: 



County Elementary 




City/Special Elementary 




Schools 


992 


Schools 


383 


County Secondary 




City/Special Secondary 




Schools 


207 


Schools 


68 


County Combined 




City/Speciat Combined 




Schools 


61 


Schools 
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The schools in each category were assigned a number, then a 
table of random numbers was used to select ten percent of the 
schools in each category. The principal of each of these schools, 
a tola! of 176 principals, was thus selected as a participant in 
the Phase Two survey. Since principals were considered a more 
homogeneous group with regard to opinion on educational is- 
sues than school board, city council, or county court members, 
no attempt was made to assure that principals contacted in 
Phase One of the survey would not be asked to participate in 
Phase Two (i.e., this assurance probably would not have resulted 
in a significant broadening of the range of principal opinion 
expressed in the survey). 

To obtain a sample of teachers for participation In Phase 
Two, the stratification of schools prepared for the process of 
selecting principals was used again. The same randomizing 
procedure was folfowed to select one-quarter of the schools 
within each of the six categories. Then the listing of teachers for 
each of these 433 schools was consulted. If the staff of a given 
school consisted of 25 or fewer teachers, a table of random num- 
bers was employed to select one teacher from the school for 
participation in the Phase Two survey. Two teachers were se- 
lected from each school having 26 or more teachers. Two 
teachers — one elementary and one secondary — were selected 
from each combined (grade spread 0-12 or 1-12) school regard- 
less of staff size. This process resulted in a sample of 547 teach- 
ers for Phase Two participation. 

Survey instruments were mailed to the 1453 individuals in 
the survey sample (representing city council members, county 
court members, school board members, superintendents, prin- 
cipals, and teachers) during November and December 1973. A 
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follow-up mailing to non-respondents was undertaken in Jan- 
uary 1974. 

Treatment of Data 

All survey instruments were coded by county, type of school 
system, group (city council, teacher, etc.), election or appoint- 
ment to office (if applicable), school organizational level (ele- 
mentary, secondary, combined; If applicable), Grand Division of 
the Slate (see Appendix Q), and Planning and Development Re- 
gion (see Appendix C). Responses were keypunched, then 
processed using an IBM 360/65 computer. Responsibility for 
keypunching and programming was assumed by Mrs. Alice 
Beauchene, programmer, at the University of Tennessee Com- 
puting Center. 

III. SURVEY FINDINGS 

THE RESPONDENTS 

Approximately 54 percent of the individuals asked to rank the 
ten education Issues identified in this study returned survey 
Instruments. Unfortunately, some of the replies were not usable 
because respondents had not ranked all items, or had used a 
ranking system that yielded results not compatible with the 
results of the 1 to 10 ranking called for in the instructions. Con- 
sequently, data analyses were based on the replies of 736 per- 
sons, approximately 51 percent of the 1453 Individuals In the 
survey sample. Table 2 shows the number and percentage of 
individuals in each of the six survey groups who submitted 
usable forms. Professional opinion was rather well represented; 
lay opinion was rather poorly represented. 

TA8LE 2. NUMBER AND PERCENTAGE OF INDIVIDUALS 
IN EACH Of SIX GROUPS RESPONDfNO TO SURVEY 



CITY COUNTY SCHOOL SUPERIN- PRINCN 
COUNCIL COURT BOARD TENDENTS PAIS TEACHERS TOTAL 



Number of 
Respondents 


28 


67 


113 


107 


104 


317 


736 


Tola! in 
Samptd 


102 


190 


292 


146 




547 


1453 


Response 
Percentage 


27.4 


35 2 


387 


73 3 


59 1 


58,0 


507 
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As wilt be shown, the extent of agreement between these 
groups of Tennesseans regarding the relative importance of the 
Issues utilized in this survey was^o great that issue rankings of 
several groups could be comb)ffed without altering general con- 
clusions based on the d^ta. Therefore, the key questions f^ 
assessing the representajiveness of survey response became J' 

I 

(1) How adequately/ was the State represented geographic- 
ally? and »^ 

(2) How adequately were county and city/special distriw 
school systems represented? j 

In respon<^e to the first question, replies were received from 
individuals in aii of the State's ninety-five counties. An average 
of eighty-two' replies was obtained from each of Tennessee's 
nine Planning and Development Regions (a map showing the 
boundaries of these Regions is included in Appendix C). In 1973, 
43.1 percent of Tennessee's public schools were located in East 
Tennessee, 32.1 percent in Middle Tennessee, and 24.8 percent 
in West Tennessee (Banta, 1973, p. 64). This distribution may be 
taken as indicative of the distribution of population throughout 
the State, at least for the purposes of this study. Percentages of 
survey returns from the Grand Divisions of the State (see Ap- 
pendix B for a listing of counties In each Grand Division) approx- 
imated these figures closely: 43 percent of the replies came from 
East Tennessee, 30.8 percent from Middle Tennessee, and 26.2 
percent from West Tennessee. 

In 1973, 73.5 percent of Tennessee's public schools were in 
county systems and 26.5 percent were in city/special district 
systems (Banta, 1973, p. 146). In the present study, then, city/ 
special district systems were slightly over-represented. Of the 
736 survey instruments returned 244, or 33.2 percent were from 
individuals associated with city/special district systems, and 
492, or 66.8 percent were from individuals associated with 
county systems, 

COMPOSITE RANKING OF ISSUES 

Rankings assigned to each issue by individuals in each of the 
six groups surveyed (i.e., city council, quarterly county court, and 
school board members; superintendents, principals, and teach- 
ers) were summed, then averaged (see Appendix E). By assigning 
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the rank of 1 to the issue given the lowest average ranking by a 
particular group, and continuing to number that group's issues 
through 10 (the issue with the highest average ranking), a sum- 
mary ranking of the ten critical education issues was caicuiated 
for each group. Table 3 presents the summary rankings for the 
six groups of Tennesseans incfuded in the survey. 

A composite ranking for all Tennesseans surveyed was calcu- 
lated in the following way. Sums of rankings for each issue 
across six groups were weighted', then summed and averaged. 
The average rankings were then ranked from 1 to 10, as above, 
to yield the composite ranking for all groups of Tennesseans 
surveyed. This Tennessee composite' ranking appears as the 
last column of Table 3. 

Listed in order from most important to least important the 
ten issues were: 

(1) Financing education— including salaries 

(2) Teacher competence 

(3) Vocational education programs 

(4) Discipline 

(5) Lack of concern by pupils, staff, parents, and public 

(6) Size of classes — overcrowding and overloaded staff 

(7) Improvement of general curriculum 

(8) Inadequate facilities 

(9) Special education programs 

(10) Administrative reform and/or reorganization 



*More (eachers (a totat of 317) returned survey Inslrumenls than any other 
group. In order to make the cor^tribut^on to the composite ranking of each of 
the other five groups equaf to the fnput of teachers, the raw sums of Issue 
rankings for these five groups had to be weighted using a factor equivalent to 
317/number of respondents In the group. That \%, the sum of rankings for each 
issue within the city council grouping was multiplied by 317/28; for county 
court members the factor used was 317/67; for school board members 317/113; 
for superintendents 317/107; and for principals 317/104. To obtain an average 
for each Issue, the total of weighted sums of rankings across the six groups 
was divided by 31 7 x 6 or 1 902. 
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TABl€ 3. SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION 
ISSUES FOR Six GROUPS OF T6NNESSEANS 





CITY 
COJNC 


CNTY 
COURT 


SCH 
BRD 


SUPT 


PRINC 


TCHR 


TENNESSEE 
COMPOSITE 


TCHR COMPETENCE 


1 


3 




3 




5 


2 


VOCATIONAL EOUC 


2 5 


2 


1 


2 


7 


6 


3 


CLASS SIZE 




7 


5 


9 


3 






GEN CURRICULUM 


6 


6 


7 


6 


5 


7 


7 


SPECiAl EOUC 


9 


10 


9 


7 


6 


8 


9 


FINANCING EDUC 


4.5 


5 


3 


1 


1 


1 


1 


OlSCtPUINE 


25 


\ 


4 


6 


4 


3 


4 


APATHY 


4.5 


4 


6 


5 


6 


4 


5 


FACILITIES 


7 


8 


8 


4 


9 


9 


8 


ADMIN REFORM 


10 


9 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


20 


67 


113 


107 


104 


317 


736 



Coelftdenl of concordance : .693 

Ch[ square. 9dr ; 37,439 

Probability of independence : .000 

The statistics below Table 3 indicate that there was substan- 
tial agreement among these six groups of Tennesseans with 
regard to the relative importance of the specified set of ten edu- 
cation issues. To calculate the degree of association between 
three or more sets of rankings, the nonparametric statistic Ken- 
dall's coefficient of concordance (W) may be used (Siegel, 1956, 
pp. 229-238). Since ten issues were ranked, the chi square 
distribution was utilized to calculate the significance of W. Ac- 
cording to tabled values, if chi square in this case exceeded 
21.67, W could be considered significant at the .01 level. If W 
were significant, one would reject the hypothesis that the six sets 
of rankings were independent. Obviously, the calculated value of 
chi square (37.439) exceeded the tabled value of 21.67, thus the 
probability that the six sets of rankings were independent was 
negligible. The conclusion is that the sets of rankings were 
highly associated. 

This high degree of association between sets of rankings was 
maintained when the rankings obtained for clusters of related 
groups were examined (see Table 4). The rankings of city council 
and county court members were combined (with raw sums of 
city council rankings being weighted to make them equivalent 
in value to the input from county court members) to form a 
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"governmental" or "fiscal authority" cluster. School board mem- 
bers stood alone as "policy makers." A cluster of "school 
professionals" was formed by combining the rankings of superin- 
tendents, principals, and teachers (with raw sums of superin- 
tendent and principal rankings being weighted to make them 
equivalent in value to the input from teachers). 

TABLE 4. SUMMARY RANKINGS OF TEN ISSUES FOR 
THREE CLUSTERS OF TENNESSEANS 



FiSCAL POLICY SCHOOL 

AUTH0RI1 lES MAKERS PROFESSIONALS 



TCHR COMPETENCE . \ 2 2 

VOCATIONAL EOUC 3 1 4 

CLASS StZE 6 6,3 

GEN CURRICULUM 7 7 7 

SPECIAL EOUC 9 9 9 

FINANCING EOUC & 3 1 

DISCIPLINE 2 4 5 

APATHY 4 6 6 

FACILITIES .8 6 d 

AOMIN REFORM 10 10 10 



TOTAU NUMBER 

IN EACH GROUP 95 113 528 



CoeH^ciant ot concorrfanco 
Chi «qjBre, 9dr 
Probability or mdependencd 



.898 
24.23€ 
.004 



There was perfect agreement among the clusters of respond 
ents that 'improvement of general curriculum', 'inadequate facil- 
ities', 'special education programs', and 'administrative reform' 
should be ranked 7, 8, 9, and 10, respectively. However, with 
respect to 'financing education', the issue ranked first in the 
composite, there was a difference of opinion. School profession- 
als saw this as the most important Issue, but fiscal authorities 
ranked it fifth. School board members — the policy makers — 
took a middle position, ranking financing third. 

Policy makers and school professionals differed In their per- 
ceptions of the importance of 'vocational education' as an issue. 
The school board members considered it the most important 
issue of all, white school professionals ranked it fourth. Fiscal 
authorities, in ranking 'vocational education' third, appeared 
closer to school professionals here than to policy makers. 
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Fiscal authorities differed from school professionals, however, 
in the ranking given to 'discipline*. Fiscal authorities considered 
'discipline' second only in importance to 'teacher competence'. 
School professionals were only moderately troubled by discipline 
problems, giving 'discipline' a ranking of 5. With their ranking of 
4, school board members were closer in this instance to the 
school professionals. 

'Size of classes' proved to be a matter of more concern to 
school professionals (who ranked it 3) than to policy makers 
(who ranked it 5) or to fiscal authorities (who ranked it 6). *Lack 
of concern' by pupils and teachers, parents, and the public was 
viewed as a greater problem by fiscal authorities (ranking of 4) 
^than by policy makers or school professionals (both ranking It 6), 

While there was substantial agreement among the clusters 
that 'teacher competence' was a top Issue, fiscal authorities 
tended to be slightly more concerned about it (with a ranking of 
1) than school board members or school professionals (with 
rankings of 2). 

GROUP INTERCORRELATIONS 

Analysis of the degree of association among the rankings of 
the various groups included in the present survey would not be 
complete without a look at correlations between pairs of groups. 
The Spearman rank correlation coefficient (r,) was employed to 
measure the extent of association between the fifteen pairs of 
survey groups (Siegel, 1956, pp. 202-213). Table 5 presents the 
group intercorrelations. With ten issues to be ranked, r, must 
equal or exceed .564 to be significant at the .05 level, and .746 
to be significant at the .01 level. 

TABLE 5. IMTERCORREUTIONS Of SETS OF RANKINGS 
FOR SIX GROUPS Of TENNESSEANS 





CITY 
COONC 


CNTY 
COURT 


SCH 
BRD 


SUPT 


PR INC 


TCHR 


CITY como 


1.000 




.922*" 


,558 


.631* 


.619* 


CNTY COURT 




1.000 




.4)9 


.526 


.564* 


SCH BRD 






1 000 


.637» 


.697' 


.673» 


SUPT 








1.000 


.358 


,261 


PRIN 










1.000 


.879" 


TCHR 












1.000 



•Sigtiilicant al the .05 level 
*"Si9ni(IC8nt at the .001 level 
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Table 5 shows a very high level of agreement between city 
council and county court members — the 'fiscal authorities' — 
with regard to the ranking of the ten survey issues. Fiscal authori- 
ties and 'policy makers' — school board members — certainly 
saw eye-to-eye on these issues. The other pair showing a highly 
significant degree of agreement consisted of principals and 
teachers. 

Good agreement existed between teachers and all other 
groups except superintendents. Principals tended to think most 
like teachers, then school board and city council members; but 
not so much like county court members or superintendents. 

By far the most divergent of the six groups surveyed was the 
group of superintendents. At least with regard to the relative 
importance of the ten education issues used in this survey, 
superintendents^ views were quite different from those of teach- 
ers, principals, and county court members. The coefficient of 
correlation between superintendents' rankings and rankings of 
city council members approached significance, but the only 
group with which superintendents showed substantial agreement 
was the one containing school board members. With reference to 
specific issues, superintendents tended to see 'inadequate fa- 
cilities* as a more important issue (ranking of 4 compared with 
composite ranking of 8), and 'size of classes* and 'discipline' as 
less important issues (rankings of 9 and 8, respectively, com- 
pared with composite rankings of 6 and 4) than did the other 
groi c;'^ of respondents. 

Inlcrcorrelations for pairs of the six groups of Tennesseans 
(as reported in Table 5) revealed some differences that were lost 
when several of the groups were combined to form clusters (as 
In Table 4). The high coefficients of correlation that appear in 
Table 6 are indicative of the significant degree of agreement 
between the three clusters of Tennesseans concerning the im- 
portance of various education issues. 



TABLE 6. INTERCORRELATIONS OF SETS OF RANKINGS FOR 
THREE CLUSTERS OF TENNESSEANS 



FISCAL AUTHORITIES 


POLICY WAKERS 


SCHOOL PROFESSIONALS 


FISCAL AUTHORITIES 1,000 


.890'" 


.757** 


POLICY MAKERS 


KOOO 


.890**» 


SCHOOL PROFESSIONALS 




1.0OO 





"Significant n the .01 level 
•"Significant at ihe .001 Jevel 
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SUBGROUPS 



Elected and Appointed Scliool Boards 

In 1973-74 election by popular vote constituted the principal 
method for selecting school board members in 72 county systems 
and 28 city/special district systems (Tennessee School Boards 
Association, 1973a), In 23 counties the majority of school board 
members was appointed, either by the quarterly county court or, 
as in the Metropolitan Nashville-Davidson County system, by the 
Mayor with approval from the Metro Council, In 23 city/special 
district systems appointment by the city council or commission 
was the chief means of filling school board positions. Proponents 
of election and advocates of appointment both argue that theirs 
is the better means for selecting school board members who are 
competent and responsive to the educational needs of the com- 
munity. Was there a difference between elected and appointed 
school board members in their abilities to sense prevailing local 
sentiment concerning key education issues? Did the two groups 
differ significantly from each other in their rankings of the ten 
survey issues? Table 7 suggests some answers. 

TABLE 7. SUMMARY RANKINGS OF TEN EDUCATION ISSUES FOR 

HLecreo and appointed school boapd members 

AND FOR FISCAL AUTHORITIES 





ELECTED SCH BRD 


APPOINTED SCH BRD 


FISCAL AUTHORITIES 


TCHR COWPETENCE 


15 




1 


VOCATIONAL EOUC 


1.5 


1 


3 


CLASS SIZE 


4 


S 


6 


QEN CURRICULUM 


& 


6 


7 


SPECTAL EDUC 




6 


9 


riNANCINQ EDUC 


3 


4 


5 


DISCIPLINE 


5 


3 


? 


APATHY 


6 


7 


4 


FACILITIES 


7 


9 


6 


ADMIN REFORM 


10 


10 


10 


TOTAL NUMeER 






95 


IN EACH GROUP 


72 


41 



' fcetweei fankings of elecled and appointed school board 

* membe^i: .900 {significant at .001 levels 

' between rankings of elected school board members and 

Tenresse^ composite: .912 (significant at .001 level) 
' between rankings of appointed school board members and 
' Tennessee composite; ,867 {significant ai .001 level) 

* between rankings of elected school board members and 

* fiscal ^uthor(ties: .845 |sig^jJ;canl &\ .001 level) 

r between rankings of appointed school board merr^bers and 

* fiscal auihofittes: .679 («ignificar>! al .001 level) 
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The statistics indicate that the thinking of elected and ap- 
pointed school board members was strikingly similar, at least 
with regard to the relative importance of the ten survey issues. 
Rankings for both sub-groups were in substantial agreement with 
the composite ranking for Tennessee and responses of both 
groups were significantly related to responses of fiscal authorities 
— the city councils and county courts responsible for appointing 
almost one-third of the school boards. The differences that did 
exist between school board members and other groupings were 
due to: (1) less concern about financing education among both 
elected and appointed school board members than the Tennessee 
sample in general, (2) stronger feelings about 'lack of concern' 
on the part of fiscal authorities than by appointed school board 
members, and (3) a higher ranking for 'discipline' by fiscal author- 
ities than by elected school board members. 

Elected and Appointed Superintendents 

According to a 1973 research report of the Tenncccee School 
Boards Association (T.S.B.A., 1973a) 75 superintendents of county 
school systems were elected to their positions by popular vote; 
all superintendents of city/special district systems were ap- 
pointed by their local boards of education, and 17 county super- 
intendents were appointed by their quarterly county courts (p.4). 
For the purposes of this study, then, superintendents were di- 
vided into three sub-groups: 'county elected', 'county appointed', 
and 'city appointed'. 

The issue of election vs. appointment of superintendents came 
up in several of the replies to the initial survey which sought to 
identify critical concerns for use in the second phase of this 
study. Was there a difference in thinking between superintendents 
who were given their jobs by vote of the people and those vjho 
were appointed? Was there a difference between superintendents 
appointed by school boards, and those appointed by county 
courts? Did any one of the sub-groups tend to reflect more 
accurately than the others the general opinion of Tennesseans 
concerned with education? How did the suo-groups of superin- 
tendents compare with other groups of Tennesseans in their 
perceptions of the critical issues used in the present survey? 
Data presented in Tables 8 and 9 provide some answers to these 
questions. 

The opinion of superintendents regarding relative issue priori- 
ties was less in accord with the thinking of the rest of the Ten- 
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nesseans surveyed than was the opinion of any other group 
Included in the survey sample. Table 9 provides an indicallon of 
the relationship between the sub-groups of superintendents and 
the other survey groups. 

TABLE 8. SUMMARY RANKINGS OF TEN EDUCATION ISSUES FOR 
EtECTEO ANO APPOINTED COUNfY SUPERINTENDENTS 
AND APPOINTED CITY SUPERINTENDENTS 





COUNTY 
ELECTED 


COUNTY 
APPOINTED 


CITY 


TCHR COMPETENCE 


3 


3 


2 


VOCATIONAL EOUC 


2 


4 


3 


CLASS SIZE 


7 


6.5 


9 


r»EN CURRICULUM 


9 


8 


4 


SPECIAL EDUC 


d 


65 


5 


FINANCING EDUC 


1 


1 


1 


DISCIPLINE 


6 


9 


8 


APATHY 


5 


2 


7 


FACILITIES 


4 


5 


6 


AOMIN REFORM 


10 


10 


10 


TOTAL NUMBER 
BACH GROUP 


53 


16 


38 


TABLE 9. SUPERINTENDENT SUB-GROUP INTERCORRELATIONS AND SELECTED 
CORRELATIONS BETWEEN RANKINGS OF ELECTED AND APPOINTED COUNTY 
SUPERINTENDENTS AND APPOINTED CITY SUPERINTENDENTS AND 
OTHER GROUPS OF TENNESSEANS 




County Elected 


^ounfy Appointed 


Ctty Appointed 


County Elected 


VOOO 


.893* 


.684* 


Cooftiy Appoinied 




1.000 


J65*« 


City Appoinied 






1.C00 


Tennessee Composite 




.693' 


.648* 


School Board 


.762" 




.576^ 


County Court 


.564* 


331 


.310 


Principals 


ACT 


,373 


.467 


Teacher* 


.491 


.391 


.225 



♦Significant at .05 fevel 
•'Significant at .01 level 
•"Significant ai .001 {evel 



Good agreement existed between elected and appointed 
county superintendenls, the chief difference between them being 
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a greater concern about 'discipline' among the eiected sub-group. 
City superintendents tended to think more like the appointed 
county superintendents than like the elected ones, but even so, 
significant differences were apparent. 'Improvement of general 
curriculum' and 'special education programs' were issues of 
more importance to city superintendents than to either of the 
county sub-groups. Both sub-groups associated with county 
systems saw 'lack of concern' as a bagger problem than did city 
superintendento. 'Discipline' was more important to elected 
county superintendents than to either of the other superintendent 
sub-groups. 

Other correlations in Table 9 Indicate that generally speaking, 
the Issue rankings of elected county superintendents were most 
like the Tennessee composite and the rankings of the other 
groups surveyed, rankings of city superintendents were the most 
different, and appointed county superintendents usually occupied 
a position in between elected county and city superintendents. 
Interestingly enough, elected county superintendents were cfoser 
in thinking than appointed county superintendents even to county 
court members — those who appointed the 'appointed' superin- 
tendents. County court members saw 'discipline' as the number 
one Issue while their appointees ranked it ninth. Both elected 
and appointed county superintendents viewed financing and 
'inadequate facilities' as more important issues than did county 
court members. 

Elected county superintendents also shared more opinions 
with school board members than did school-board-appointed city 
superintendents. City superintendents saw 'special education 
programs' as a more important Issue, 'size of classes' and 'dis- 
cipline' as less important Issues than did school board members. 
All three sub-groups of superintendents were more concerned 
about 'inadequate facilities' and less troubled about 'discipline' 
than were school board members. 

Since issue rankings by principals and teachers were so 
highly related, it was not surprising that their differences with 
superintendents were similar. In general, superintendents tended 
to view 'vocational education programs' and 'inadequate facilities' 
as more important, and 'size of classes' and 'discipline' as less 
Important than did principals and teachers. 

In short, superintendents, the most divergent of the six groups 
of Tennesseany sampled, considered 'inadequate facilities' a 
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more important issue and 'discipline' a less important Issue than 
did the other groups. Superintendents felt more strongly about 
'financing education' than did county court members, more 
strongly about 'vocational education programs' than principals 
and teachers, and less strongly about class size than principals 
and teachers. 

City superintendents, the most divergent sub-group, differed 
from their fellow superintendents in that they viewed 'improve- 
ment of general curriculum' and 'special education programs' as 
more important iss jes and 'lack of concern' as a less important 
issue than did the others. In comparison with all Tennesseans 
surveyed, city superintendents were more concerned about im- 
proving curriculum and special education and less concerned 
about 'discipline' and class size. 

Principals of Elementary, Secondary, and Combined Schools 

Administrative responsibilities and concerns must differ some- 
what for principals of elementary, secondary, and combined 
(grades K-12 or 1-12) schools. But did principals dealing with 
each of these organizational levels differ significantly in their 
perceptions of the ten education issues utilized in the present 
study? Table 10 and its accompanying statistics were designed 
to reveal such differences if they existed. 

TABLE to. SUMMARY RANKINGS Of TEN EDUCATION ISSUES FOR PRINCIPALS 
Of EtEMENTARY, SECONDARY, AND COMBINED TENNESSEE SCHOOLS 





Elementary 


Secondary 


Comblnad 


TCHR COMPETENCE 


3 


A 


2 


VOCATIONAL EOUC 


7 


3 


\0 


CUSS SIZE 




6 


4 


G€N CURRICULUM 


5 


2 


7.5 


SPECIAL £DUC 


8 


7 


9 


FINANCING EOUC 


t 


1 


7.5 


OtSCIPLiNE 




8 


3 


APATHY 


6 


5 


1 


fAClHTiES 


& 


9 


5.5 


ADMIN REFORM 


10 


^0 


55 


TOTAL NUMBER 
IN EACH GROUP 


77 


24 


3 



r between rankings of elemGrtary and secondary 

principals: $36 (jiflnjficant at .02 level) 
r beJwecn rankings of elementary principals and 

• Tennessee composite.. .757 (sigmticant at .0^ (eveU 

r be^v^een ranVing* of secondary principals and Tennessee 

• composite: .657 (significant at .02 level^ 
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Although the percentage of respondents in each of the three 
*principar categories approximates the percentage of schools In 
each of these categories State-wide, the number of respondents 
in the 'combined' category was too smali to include in statistical 
analyses. Suffice it to say that principals of combined schools 
appeared to see Mack of concern' and 'administrative reform' as 
much more important issues than did the Tennessee sample in 
general; and these principals seemed to consider 'financing 
education' and 'vocational education* as less important than did 
all Tennesseans surveyed. 

Even though the correlations were significant between rank- 
ings by elementary and secondary principals, and between rank- 
ings by the principals and the Tennessee composite, a substantial 
amount of the vari; nee among these groups was unaccounted 
for by shared elements. Thus there may be some Interest in 
looking at the chief differences between these categories of 
respondents. 

Elementary princfpafs tended to view 'vocational education' 
as a much less important issue than did secondary principals 
and all Tennesseans surveyed. These principals were more 
concerned about 'size of classes' than secondary principals or 
the composite of all groups sampled. Secondary principals evi- 
denced more concern about improvement of general curriculum' 
than did efementary principals and all Tennesseans. 'Discipline' 
was less of a concern to secondary principals than to the other 
two groi^pings. 

Ef'^mentary and Secondary Teachers 

e(ementary and secondary teachers certainly face different 
tasks, but did they differ significantly in the way they viewed 
critical issues in Tennessee eduutilion in 1973-74? Did the rank- 
ings of issues by either group differ from the composite ranking 
for Tennessee? Table 11 and ils accompanying statistics present 
data related to these queries. 

Rankings of the ten survey issues for elementary and sec- 
ondary teachers wore significantly related, yet some substantial 
differences between the ^^o sub-groups were apparent. The 
ranking for each of the sub-groups was significantly related to 
the composite ranking for Tennessee, but the correlation for 
secondary teachers was much higher than that for elementary 
teachers. Elementary teachers viewed class size as a much more 
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important issue, and 'vocational education' as a less important 
issue than did secondary teacliers or all Tennesseans surveyed. 
Teacher competence* was a (ess important issue for both ele- 
mentary and secondary teachers than for the Tennessee sample ' 
as a whole. Secondary teachers were more concerned about 
•improvement of general curncufum' than were elementary teach- 
ers or alt Tennesseaiis sampled. Secondary teachers also dif- 
fered somewhat from the Tennessee composite ranking on the 
issue Mack of concern': secondary teachers considered it second 
in importance while the Tennessee groups in general placed 
it fifth. 

TABLE 11. SUMWAP.Y RANKINGS Of TEN EDUCATION ISSUES FOR 
ELEMENTARY AND SECONDARY TEACHERS IN TENNESSEE 





Elemeniary 


Secondary 


TCHR COMPtTENCE 


5 


5.5 


VOCATIONAL EOUC 


6 




CLASS SIZE 


1 


7 


OEM CVRR)CViVM 


9 


5.5 


Special eouo 


7 


9 


PINANCIKO EOUC 


2 


1 


OISCIPL(N£ 


3 


4 


APATHY 


4 


2 


^>C(LITJeS 


8 


6 


ADMfW REFORM 


10 


10 


TOTAL NUVBER 
IN EACH GROUP 


233 


84 



»• between ranking* of clemenfary and secondary 

' feachers; {signlJicant at ,05 level) 

r between ranking* of elementary teachers and Tennessee 

' composile: ,B72 (significant at .02 level) 

r between ranking? of secondary feachera and Tennessee 

• composite: -651 {signilicant at 01 level) 

COUNTY AND CITY/SPECIAL DISTRICT SYSTEMS 

Survey replies were coded so that each could be Identified 
with the (ype of schoof system its sender represented: (1) county, 
(2) city, or (3) special district. For purposes of the analysis city 
and special district systems were combined to form a single 
category. 

Table 12 contains a composite ranking of the ten survey 
issues across all county system groups, and five sets of summary 
rankings: one for the county court members, one for the school 
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board members, one for the superintendents, one for the princi- 
pals, and one for the teachers associated with county school 
systems. The *city council' category was not applicable since 
fiscal authority for county school systems is exercised by the 
appropriate quarterly county court (or, in some instances, by a 
county council or metropofitan council. In such cases the replies 
from members of theso bodies were placed in the 'county court' 
category). 

The composite ranking for all county system groups was 
computed as described on page 18, that is the raw sums of 
rankings for each group were weighted to make them equivalent 
in value to the input from teachers (the largest group), then the 
weighted sums were averaged and ranked from tow (rank of 1, 
most important) to high (rank of 10, least important). 



TABLE 12 SUMMARY AND COMPOSHE BANKINGS OF TEN EOUCATtON ISSUES FOR 
five GROUPS OF TENNESSEANS ASSOCIATED WITH COUNTY SCHOOL SYSTEMS 





CITY 
COUNC 


CNTY 
COURT 


SCH 
BRO 


SUPT 


PR)NC 


TCHR 


COUNTY 
COMPOSITE 


TCHft COMPETENCE 






2 




2 


5 




VOCATIONAL EOUC 




2 


\ 


2 


7 


6 


2 


CLASS SIZE 




7 


A 


6 


A 


1 


5 


OBN CURRICULUM 




& 


8 


6 


S 


8 


8 


SPECIAL EOUC 


o 
a 


10 


9 


7 


9 


9 


9 


FINAMCINQ EOUC 


a 
< 


s 


3 


1 


T 


2 


t 


DISCIPLINE 


o 
z 


1 


5 


9 


3 


4 


4 


APATHY 




4 


6 


5 


6 


3 


6 


FACILITIES 






7 


4 


a 


7 


7 


ADMIN REFORM 




9 


TO 


10 


10 


to 


10 


TOTAL NUMBER 
(N EACH GROUP 


0 


67 


65 


C9 


69 


i22 


492 



Coefficient of concordance : .075 

Chi SQjare, 9df : 30.382 

Probability o? independence ; .000 
f bet/*C;en COun»y composite an<J composite for Tennessee: 
963 (signiftcanJ a! .001 level) 

The highly significant coefficient of concordance indicates 
that there was a high degree of association between issue rank- 
ings of the five groups of Tennesseans connected with county 
school systems. !n order to compare the composHe ranking for 
all county groups with the composite ranking of issues for ad six 
groups of Tennesseans surveyed, a Spearman rank coefficient 
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of correlation (r,) was calculated. Since r, = .963, there was 
near-perfect agreement between the two composite rankings. 

Table 13 contains the summary and composite rankings for 
the five groups associated with city/special district school sys- 
tems. The 'county court' category was not applicable in this 
instance since fiscal authority for city and special district systems 
rests with the appropriate city council. 



TABLE 13. SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES 
FOft FIVE GROUPS OF TENNESSEANS ASSOCIATED WITH 
CITY/SPECIAL DISTRICT SCHOOL SYSTEMS 





c^ry 

COUNC 


CWTY 
COURT 


SCH 
BRD 


SUPT 


PRINC 


TCHR 


CITY 
COMPOSITE 


TCHR COWPETENCE 


1 




1 


2 


2 


5 


2 


VOCATIONAL EDUO 


25 




2 


3 


7 


7 


3 


CLASS SIZE 


6 




6 


9 


3 


2 


65 


GEN CURRtCUlUM 


B 


jg 


5 


4 


4 


6 


5 5 


SPECIAL EOUC 


9 


a 
a. 


8 


5 


6 


8 


8 


FINANCIf^G EOUC 


4.5 


a 
< 


3 


1 


1 


1 


1 


DISCIPLINE 


2 5 


o 
z 


4 


8 


5 


3 


4 


APATHY 


4.5 




7 


7 


8 


4 


7 


FACILITIES 


7 




d 


6 


9 






AOMIN REFORM 


10 




10 


10 


10 


10 


to 


TOTAL NUMBER 
IN EACH GROUP 


28 


0 


48 


38 


35 


95 ^ 


244 



Coefficient of concordance : .665 

Ch^ square, 9dl 29 931 

Probability of Independence t .000 
f be'ween City composite and composite for Tennessee: 

.951 fslpnificant at XOi level) 
r between City composite and County composite: 
'» .927 (significant at ,001 level) 

The statistics below Table 13 indicate a very high level of 
agreement (1) among the five groups associated with city/speciat 
district school systems, (2) between groups associated with city 
systems and all groups surveyedr and (3) between groups asso- 
ciated with city systems and those connected with county 
systems. Though the differences between County and City com- 
posite rankings were very small, there did seem to be an indica- 
tion that groups associated with city/special district systems 
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were more concerned about 'improvement of general curriculum' 
while those associated with county systems were more con- 
cerned about Inadequate facilities'. 

GRAND DIVISIONS OF TENNESSEE 

Each survey reply was coded to indicate the Grand Division 
of the State ^which its sender represented (see Appendix B for 
a listing of counties in each Grand Division). Tables 14, 15, and 
16 contain summary rankings, and composite^ rankings of the ten 
survey issues across all groups, for (1) East Tennessee, (2) 
Middle Tennessee, and (3) West Tennessee. In Table 15 note that 
the two governmental groups (city council and county court) 
were combined because there were fewer than five responses 
from city council members. The computed statistics indicate 
high levels of agreement among the groups surveyed in each 
of the Grand Divisions. 



TABLE 14. SUMMARY AND COMPOSITE RANKINGS OF TEN EOUCATiON ISSUES 
FOR SIX GROUPS IN EAST TENNESSEE 





CITY 
COUNC 


CNTY 
COURT 


SCH 
8R0 


SUPT 


PRINC 


TCHR 


EAST 
COMPOSITE 


TCHR COMPETENCE 


2 


3 


2 


3 


3 


3 


2 


VOCATIONAL EOUC 


1 


1 


1 


2 


7 


6 


3 


CUSS SUB 


4 


7 


5 


8 


4 


2 


5 


GEN CURRICULUM 


7 


6 


7 


4 


2 


8 


65 


SPECIAt EOUC 


6 


85 


a 


7 


9 


9 


9 


FINANCING EOUC 


6 


2 


3 


1 


1 


1 


1 


DISCIPLINE 


5 


S 


4 


9 


6 


6 


6.5 


APATHY 


3 


4 


6 


G 


5 


4 


4 


FACILITIES 


10 


85 


9 


S 


8 


7 


8 


ADMIN REFORM 


9 


10 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


11 


23 


51 


41 


51 


139 


3t6 



Coefficient of concordance .698 
Chi square, 9df 37.665 
Probability of Independence : .000 
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TABLE 15. SUMMARY AND COK^POSITE PANK^GS OF TEN EOUCATION ISSUES 
FOR FIVE GROUPS IN MIDDLE TENNESSEE 





CITY 
COUNC 


CNTY 
COURT 


SCK 
BRD 


SUPT 


PRINC 


TCHR 


MIODLE 
COMPOSITE 


TCHR COMPETENCE 




3 


4 


2 


2 


5 


2 


VOCATIONAL EOUC 


c 


2 


t 


3 


6.5 


6 


4 


CLASS SI2E 


J 


7 


25 


9 




2 


5 


















GCN CURRICULUM 


e 

3 


6 




6 


4 


7 


7 


SPECIAL EDUC 


O 


10 


9 


7 


6 5 


8 


9 


FINANCING EDUC 


i 


4 


5 


1 


1 


3 


1 


OlSCIPllNE 


"O 

« 
c 


1 


25 


S 


3 


1 


3 


APATHY 


!5 
E 


$ 


7 


4 


6 


4 


6 


FACILITIES 


8 


9 


6 


6 


9 


9 


8 


AOMIN REFORM 




6 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 




25 


33 


38 


29 


102 


227 



Coefficieni of concordancft .689 
Ch( squard, 9dr : 30.106 

Probability of Independence .000 



TABLE 16. SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES 
FOR SSX GROUPS IN WEST TENNESSEE 





CtTY 
COUNC 


CNTY 
COURT 


SCH 
BRD 


SUPT 


PRINC 


TCHR 


WEST 
COMPOSITE 


TCHR COMPETENCE 


1 


4 


1 


5 


4 


6 


2 


VOCATIONAL EDUC 


4 


3 


2 


2 


9 


6 


4 


CLASS SIZE 


5 


5 


6 


8 


2 


2 


5 


GEN CURRICULUM 


9 


8 


5 


6 


6 


7 


7 


SPECIAL EDUC 


8 


7 


8 


7 


8 


6 


9 


FINANCING EDUC 


3 


6 


3 


1 


1 


1 


1 


DISCIPLINE 


2 


1 


4 


9 


3 


3 


3 


APATHY 


6 


2 


7 


4 


5 


4 


6 


FACILITIES 


7 


9 


9 


3 


7 


9 


8 


ADMIN REFORM 


10 


10 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


16 


20 


29 


28 


24 


76 


193 



Coefficieni of concordance .605 
CM square. 9df : 32.655 

Probability of Independenct : .000 
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fntercorrelations for sets of composite rankings for East, 
Middle, West, and ail Tennessee are reported in Table 17. All 
coefficients were highly significant, indicating substantial agree- 
ment throughout th& Sprite concerning the relative importance of 
the survey issues. Note the perfect relationship between opinions 
in West and Middle Tennessee. Although these differences were 
slight, it could be mentioned that groups in East Tennessee were 
less concerned about 'discipline' (ranking of 6.5 vs, 3 for Middle 
and West Tennessee) and more concerned about Vocational 
education* (ranking of 3 vs, 4) and lack of concern' (ranking of 
4 V8- 6) than were groups in Middle and West Tennessee. 



TABLE t7. INTERCORRELATIONS OF COMPOSITE RANKINGS FOR 
EACH OF THREE GRAND DIVISIONS AND ALL TENNESSEE 



East 


Middle 


V/est 


A]} Tennessee 


€aM 1,000 






,927* 


Middle 


).000 


f.OOO"* 


.975*** 


West 




1.000 


.975' 


All Tennessee 






1.000 




'* 'Significant at Ihe .001 level 






PLANNING 


AND DEVELOPMENT REGIONS 





Each survey reply was coded to Identify the sender as a 
resident of one of Tennessee's nine Planning and Development 
Regions (see Appendix C for a map showing the boundaries of 
the Regions). Tables A-1 through A-9 in Appendix D present sum- 
mary and composite rankings of issues for each Region. Note 
that in cases where fewer than five responses were available for 
a given group, certain groups were combined. Intercorrelations 
between sets of rankings for the six proups of Tennesseans 
surveyed (see Table 5) indicated that agreement was high be- 
tween city council members, quarterly county court memberSi 
and school board members. Where necessary, two or perhaps 
three of these groups within a given Planning and Development 
Region were combined to yield a group of more than five re- 
spondents. Likewise, the significant correlation between superin- 
tendent opinion and school board opinion made it feasible to 
combine responses in these two categories when one contained 
fevv^r than five replies. Finally, principal-teacher agreement was 
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quite high, so in one instance these two groups were combined. 
Coefficients of concordance computed for the summary rank- 
ings of groups in each Planning and Development Region re- 
vealed a high degree of consensus among groups in all Regions. 
In two instances the probability of independence was approxl- 
mateJy .02, but in most instances that probability was ,001. 

Table 18 contains coefficients of correlation between com- 
posite rankings for each of the Planning and Development Re- 
gions and the composite ranking for all Tennesseans surveyed. 
Opinion in each of the nine Regions approximated rather closely 
that prevailing throughout the State, 

TABLE 18. CORRELATJONS BETWEEN COMPOSITE RANKINGS FOR EACH OF NINE 
PIANNINO AND oeVELCPMENT REGIONS AND THE 
COMPOSITE RANKING FOR TENNESSEE 



First Tennesseo 


,866"* 


Ea9i Tennessee 


.951"' 


Southeast Tennessee 


.939"" 


Upper Cumberland 


.709" 


Midcumbariand 


,975"' 


South Cer^tra) Tennessee 


,S5*'" 


Northwest Tennessee 


,939'" 


Southwest Tennessee 


.963"' 


Memphis OeJia 


745" 



"Sigafffcant at ,011 level 
'"Significant at .001 level 

ISSUE RATINGS 

Survey participants were asked to indicate in two ways the 
relative importance of the ten education issues included on the 
reply card. All survey findings reported to this point have been 
based on the ranking of issues from 1 to 10. 

As a check on both the validity of the ten-item listing (i.e., 
were all ten issues really 'critical issues in Tennessee education' 
as defined by high 'importance* ratings?) and on the reliability 
{or reproducability) of respondents* rankings, a second scale 
appeared on the reply form which gave respondents an opportu- 
nity to rate issues *A' (of critical importance), *8' (of some im- 
portance), or (of little or no importance). Table 19 presents 
the data obtained from the A, B, and C ratings. 
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TABLE 19. PERCENTAGE OF ALL SURVEY PARTICIPANTS 
RATING ISSUES A, B, OR C 





A 


B 


TCHR COMPETENCE 


55.5 


as 9 

09.4C 


VOCATIONAL EOUC 


S6.5 




CLASS SIZB 






G£f4 OiJRRlC^iJI iJM 


30.1 


92. D 


SPECIAL eouc 


3o.e 


52.3 


FINANCINQ EOUC 


57.1 


33.5 


DiSCiPLiNE 


51.7 


37.2 


APATHY 


49.2 


36.6 


FACiLITieS 


34.5 


43.6 


ADMIN REFORM 


19.4 


47.9 



(Ro^v totals should equal 100% but may not dud to rounding.) 

The listing of issues for the survey appears to possess an 
acceptable degree of validity, i.e.» the issues really were im- 
portant to a majority of the respondents. Even Issue # 10, 'admin- 
istrative reform and/or reorganization', (which admittedly was 
too broad and vague a term to convey the essential nature of 
several conlrovefsial issues subsumed by the category) was 
considered of at least 'some' importance by more than two-thirds 
of the survey respondents. At least 30 percent of the respondents 
considered all issues except # 10 *of critical importance'. More 
than 50 per cent considered *teacher competence'. Vocational 
education', 'financing education', and 'discipline' to be 'of critical 
importance'* When 'B' ratings were includod in the analysis, 
roughly 80 per cent of the respondents considered all issues 
except # 10 to be of at least *some* importance. 

An indication of the reliability of the survey instrument can be 
obtained by ranking the percentages in column W (items rated 
'of critical importance') of Table 19 from highest (rank of 1) to 
lowest (rank of 10), then computing a Spearman rank coefficient 
of correlation, between this ranking and the composite ranking 
for ail Tennesseans surveyed. When this computation was per- 
formed r. = .952, an exceedingly high level of *'test-retest" 
reliability. 

A similar indication of reliability was obtained when percent- 
ages in the *C' column were ranked from lowest, with a rank of 1 
(thus of most Importance because the fewest respondents con- 
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sidered the Issue *of little or no importance') to highest, rank of 
10, then correlated with the Tennessee composite ranking. In 
this instance r, = .916. 

PARALLEL FORMS OF THE SURVEY INSTRUMENT 
To minimize the possibility thai some issues would receive 
high rankings (and ratings) simply because they appeared first, 
or early, in the listing, two forms of the survey instrument were 
prepared: one printed on yellow paper, listing the issues 'teacher 
competence* first and 'administrative reform' last; and one printed 
on blue paper, reversing the order of all issues. TablQ 20 presents 
the summary rankings of issues on the two forms. 

While the coefficient of correlation between rankings on the 
two forms was significant, it was not hiah, and it should have 
approached 1.0 if order of presentay5n fiad had no effect on 
respondents' rankings. Actually, it isfevi^nt from inspection of 
Table 20 that issues were ranked Higher when they appeared 
first, or early, on the form, and lower) when they appeared lower 
in the listing. Presumably only ltejrv# 10, 'administrative reform', 
would have been ranked 10 regardless of its position in the listing. 

TABLE 20. SUMMARY RANKINGS OF TEN EDUCATION ISSUES 
APPEARING IN REVERSE ORDER ON TWO FORMS 





Yellow Form 


Blue Form 


TCHR COMPETENCE 


1 


6 


VOCATIONAL EDUC 


4 


5 


CLASS Size 


3 


4 


GEN CURRICULUM 


7 


a 


SPeCtAL EDUO 


8 


9 


FINANCING EDUC ^ 


2 


1 


DISCIPLINE 


5 


2 


APATHY 


6 


3 


FACILITIES 


9 


7 


AOMIN REFORM 


fO 


10 


TOTAL IN GROUP 


377 


359 



^» betv^een rankings on yellow and b1ud forms: .685 (significant at .02 level) 

Perhaps the most important information revealed in Table 20 
is the remarkable divergence in rankings given to 'teacher com- 
petence' on the two forms: 1 when it appeared first on the survey 
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instrument, 6 when it appeared last. In the computation of the 
composite ranking of items for all Tennesseans surveyed it was 
noted that 'financing education' received the rank of 1 by a very 
small margin over 'teacher competence*. Apparently 'teacher 
competence' would have been the number one issue (probably 
due largely to its position as the first item on the yellow form) in 
the Tennessee composite ranking if there had been no blue form. 

Observation of the apparent effects of order of presentation 
certainly strengthens the rationale for using more than one item 
order in a survey involving rankings. Ideally, all possible orders 
should be used to counteract the effect entirely. 

OTHER ISSUES SUGGESTED BY RESPONDENTS 

In addition to the ten critical education issues listed on the 
survey instrument, space was provided to give respondents an 
opportunity to list other concerns. About 12 percent (90) of the 
736 respondents used the space for additional remarks. This 
limited usage, plus the relatively smalf number of new issues 
listed, strengthened the conclusion that the ten issues utilized 
in the survey were the ones of most concern to Tennesseans 
closely associated with the educational process in 1973-74. As 
a matter of fact, the content of remarks appearing on the Phase 
Two reply card bore a striking resemblance to that appearing 
on the Phase One instrument^ from which the ten Phase Two 
items were derived. 

Fewer than 20 percent of the comments added by respondents 
could be classified as new concerns not covered directly in the 
listing printed on the reply card. Four individuals mentioned 
court-ordered busing to achieve racial balance as a crucial issue. 
The need for a more honest, dedicated approach to integration 
was also mentioned. Three individuals were concerned about 
pupil transportation — overcrowded buses and curtailed service 
due to the fuel shortage. Four individuals expressed concern 
about counseling and guidance programs: more counselors at 
all levels were needed, elementary guidance specialists were 
needed to initiate elementary guidance programs, counselor com- 
petence should be upgraded. Needs for other specialists such as 
librarians and reading teachers were expressed. 

Individualization of instruction was an issue which might have 
been subsumed within either of the listed issues Improvement 
of general curriculum' or 'teacher competence*. Perhaps some 
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respondents perceived individualization as a part of one of those 
categories. However, several of those who wrote in responses 
probably had in mind the need to individualize instruction when 
they mentioned 'need for more teaching materials and equip- 
ment', *chi!dren should not be retained in primary grades', and 
'need for more innovative programs'. 

Teachers were concerned about retirement plans and other 
benefits, which were not really included in the listed issue 'financ- 
ing of education* although they are closely linked with salaries. 

Other concerns listed included school opening and closing 
times, social change, community use of school facilities, and the 
compulsory school attendance requirement. At least one indi- 
vidual expressed the need for each of the following: kindergarten, 
better textbooks, religion in the classroom, and comprehensive 
surveys of the total educational program in each county. 

At least 80 percent of the comments added to the survey reply 
card could be classified as remarks about the fen listed issues. 

Financing education was seen as an overarching issue: if 
sufficient funds were available, most of the other "issues*' would 
be resolved. (Several respondents whose replies were not usable 
in the data analysis ranked only 'financing'.) Some individuals 
felt that more State and federal financing should supplement 
local efforts to support education. 

Several of the additional remarks were related to the teacher 
competence issue. Teacher competence, training, evaluation, and 
professional improvement were subsumed by this concept. Con- 
cern was expressed that too many poor and/or indifferent teach- 
ers had tenure. Temporary or emergency certification was 
responsible for some incompetent teachers. The inability of most 
teachers to properly individualize instruction disturbed more than 
one survey participant. 

A strong trend running through the responses related to 
teacher competence might be identified as deep dissatisfaction 
with current teacher preparation programs, especially the uneven 
quality of those provided by certain colleges and universities. 
Remarks indicative of this trend included 'reorganize teacher 
training programs in colleges'^ 'find some way of grading teach- 
ers other than a degree from a college*, 'future teachers need an 
intern period,' and 'training institutes needed for new teachers*. 
Related to the last comment, there was also a significant degree 
of concern about in-service training for all teachers. Improvement 
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of professional development programs sponsored by the schools 
was sought. 

Teachers expressed an interest in seeing improvements in 
standards and procedures employed in teacher evaluation. Policy 
makers and fiscal authorities wanted better ways to hold a 
teacher 'accountable' for his performance. 

Remarks about the need for career education In grades K-12 
were directly related to the stated issue vocational education 
programs. One respondent called for a 'high school course in 
career planning'. 

In connection with the discipline issue, one survey participant 
felt that a way should be devised to discipline teachers. 

None of the written responses mentioned lack of concern by 
pupils. But several noted a 'lack of concerned, dedicated teach- 
ers'. Even teachers deplored the 'lack of professionalism' within 
their own ranks. Several respondents seemed to be saying that 
the lack of concern about education on the part of parents and 
the public in general was due to poor communication: schools 
were not keeping the public informed about their programs and 
problems. 

Written remarks related to size of classes stressed the need 
for full-time aides, especially in the primary grades, to alleviate 
the teacher's work load. 

In connection with improvement of general curriculum, the 
overwhelming concern was that the quality and quantity of basic 
instruction in the three Rs must be improved, '^Prevent the need 
for remediation", said one respondent. "Teach students to read 
and to write legibly", said another. The "new" math came under 
sharp attack by several respondents. One teacher wrote, "Every 
year I get students who can't add or subtract". One survey par- 
ticipant called for training in public speaking and vocabulary for 
all students, grades 1-12. Several felt that more reading special- 
ists were needed in the schools. 

The key concern related to administrative reform involved 
removing politics from operation of school systems. Some felt 
this could be furthered by providing that all superintendents be 
appointed by boards of education. Some felt that school boards 
should b9 elected by the people rather than appointed by county 
courts. More local control of school policies and operations and 
less interference from State authorities was requested. Yet this 
sentiment appeared to conflict with such other comments as 
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"local county school boards are inefficient", "more State and 
federal financing needed," and "maintain uniform educational 
standards from county to county". There appeared to be some 
support for consolidating ail school systems within a county. 

As might have been expected, some respondents considered 
administration "top heavy" while others called for "more admin- 
istrative help". One survey participant suggested that more 
blacks should be hired for administrative positions. Several 
teachers expressed a need for Improved communication between 
teachers and administrators at the system level. 

Teachers registered a strong plea for more teaching time 
and less "administrative" responsibility. They wanted a free 
period each day for planning, and release from the bookkeeping 
chores many considered onerous. 

Other concerns related to administrative reform and/or reor- 
ganization included "extend the school year to reduce capital 
outlay", "schools getting too big; return to mid-size (500-750)/' 
and "provide for better evaluation of total programs.'* 

IV. ANALYSIS OF FINDINGS 

Statistically there was a rather remarkable degree of associa- 
tion among the rankings of ten education issues by the six groups 
of Tennesseans sampled in the present survey. There were some 
notable differences of opinion on some issues between city 
council, quarterly county court and school board members, 
superintendents, principals, and teachers; but overall, instances 
of comparative agreement far outweighed instances of compara- 
tive disagreement. The high level of association among group 
rankings made it possible to calculate and use, with some 
assurance, a 'Tennessee composite ranking* as a summary rank- 
ing representing general consensus. 

COMPARISON WITH RESULTS OF 1973 GALLUP POLL 

No attempt was made in this study to duplicate the listing of 
critical issues which resulted from the 1973 Gallup Poll of Atti- 
tudes Toward Education, Issues v/ere identified and named on 
the basis of an initial survey conducted with a sample of the 
same six groups of Tennesseans that provided the final rankings. 
Yet the education-related concerns of Tennesseans in 1973-74, 
as summarized in the Tennessee composite ranking, were quite 
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similar to those of the American public at the same time, if the 
results of the two surveys were valid. Inspection of Table 21 
reveals that direct comparisons can be made between eight of 
the ten issues listed in both surveys. Tennesseans sampled did 
not share the degree of national concern about Integration' and 
'drug use', substituting instead interest in 'vocational education' 
and ^special education'. 

TABLE 21. COMPARISON OF TENNESSEE COMPOSITE RANKING 
AND 1073 GALLUP RANKING OF TEN EDUCATION ISSUES 







(RANKING 


IN PARENTHESES) 


1973 GALLUP RANKING^ 




1973-74 TENNESSEE COMPOSITE RANKING 


(1J 


Ciftcipline 


{4J 


Discipline 


(2) 


Integration 


13) 


Vocational education prosrams 


(3) 


Financial deficiencies 


(IJ 


Financing education-including salaries 


14) 


Goo<l teacher shoitage 


12) 


Teacher competence 


(5) 


Drug use 


(9) 


Special education programs 


(6J 


School/class SiZe 


(6) 


Size of classes cvsrcrowdmg and overloaded staff 


(7) 


Poor curricufijfn 


{7) 


Improvement of general curriculum 




Parent apathy 


(5) 


Lack ol concern by pupils, stafi, parents, and public 


(9) 


Facititles 


(8) 


Inadequate facilities 


(10> School board policies 


{10) Administrative reform and/or reorganization 



iGalfup. 1973 

Statistical comparison of the two sets of rankings in Table 21 
is not really appropriate since two of the ten items are not the 
same. Yet the correlation between the two rankings as they 
stand exceeds .70, and the degree of correspondence is ob- 
viously significant. Note, for instance, that issues 6, 7, and 10 
occupy the same position in both listings, and concern about 
'facilities' was praotically the same. 

•Financing education^ including salaries' was an issue of 
greater concern to Tennesseans than to Gallup's national sam- 
ple. This can probably be explained by current National Education 
Association statistics: in 1972-73 Tennessee ranked 46th among 
the 50 states in rate of spending per pupil, and 43rd in average 
teacher salary paid (Wyngaard, 1974). 

Low salaries which do not make the State competitive with 
others in the ability to attract and hold good teachers may par- 
tially explain why the Tennessee sample ranked 'teacher com- 

42 

ERLC 



petence' higher than did the national sample. But other factors 
conlributing to the intensity of this particular concern (which 
incidentaliy came so close to 'financing education' as the number 
one concern that the two issues should probably be considered 
as co-leaders of the Tennessee listing) were identified by re- 
spondents: tenure rules that prevent dismissal of teachers who 
no longer meet the highest standards; teacher training programs 
at some colleges that do not reatisticaUy prepare pre-profes- 
sionals. for effective performance in the classroom; and Inade- 
quate in-service education programs for teachers. 

Sources of turbulence in Tennessee schools must have been 
fewer and/or less pronounced in 1973-74 than in American 
schools in general. 'Discipline* as an issue was ranked fourth by 
the Tennesseans surveyed, first by Gallup's national sample. 
'Integration' and 'drug use' were not even serious contenders for 
positions in the top ten education concerns of Tennesseans, 
though these issues were ranked # 2 and # 5 nationally. 

Concern about disruptive influences in the schools was 
replaced in Tennessee by strong feelings about the effect on the 
educational process of apathy: 'lack of concern', not just by 
parents as in the national survey, but by all associated with the 
process — pupils, staff, parents and the public. 

The Gallup survey identified 'school board policies' as a na- 
tional concern. Jn Tennessee school board policies were but a 
part of the broader issue 'administrative reform and/or reorganl- 
zation*. Other concerns subsumed by this category in the Ten- 
nessee survey included removing politics from the operation of 
school systems, consolidation of all systems within a county, and 
improving the quality of administration at all levels — State, 
system, and school. 

Although vocational education and special education pro-, 
grams received increased attention nationally in the early 1970s, 
these areas of concern were not sufficiently important to show 
up among the top ten issues in the Gallup survey. Undoubtedly 
the greater importance attached to these issues by Tennesseans 
was related to passage of important legislation in both areas by 
the Tennessee General Assembly during the year preceding the 
initiation of this study. Stale funds were provided to construct ano 
operate enough additional vocational-technical education facil- 
ities to make vocational programs accessible to all high school 
students In the State. This legislation would eventually have the 
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effect of doubling the pre-1973 need for facilities and personnel 
for vocational-technical programs. 

Legislation related to special education required that oppor- 
tunities be made available by Fall 1974 for all gifted and handi- 
capped students to be educated in the public schools. One 
possible implication of such a requirement was that in many 
schools the mildly handicapped would be placed in regular 
classrooms. Interestingly enough, neither teachers nor principals 
— seemingly the groups to be affected most by the legislation 
related to vocational and special education — saw either of 
these issues as being of more than moderate interest. 

COMPARISON WITH OTHER TENNESSEE STUDIES 

According to a survey of teacher attitude conducted by the 
Tennessee Education Association in 1973-74, Tennessee teachers 
were most concerned about such issues as (1) school financing, 
including salaries, (2) the need to lower pupil/teacher ratios and 
(3) teacher-training programs, including in-service education. 
Correspondence was significant between the top T.E.A. Issues 
and those given highest rankings by teachers in the present 
study: (1) 'financing education, including salaries', (2) 'size of 
classes' and (5) 'teacher competence'. 'Discipline' and 'lack of 
concern' rounded out the list of the five issues most important 
to teachers in the present study, but these matters were not 
mentioned in the T.E.A. survey. This was probably related to the 
fact that the T.E.A. survey was undertaken to provide input for 
a proposed legislative action program to be sponsored by T.E.A. » 
and 'discipline* and 'lack of concern' do not readily lend them- 
selves to solution by legislation. 

During the 88th Tennessee General Assembly the House 
established a 10-member bi-partisan Select Committee to Study 
Public Education in Tennessee. According to the Committee 
report (Tennessee General Assembly, 1973) impetus for the 
Committee's work was provided by "a seeming lack of confi- 
dence among parents and taxpayers in public education today" 
(p. 151). Chief areas of Committee concern were to be (1) ''quality 
of education — particularly the achievement level of basic skills 
such as reading and math", (2) "discipline", and (3) "parents' 
and students' confidence in public education'* (p. 151). 

Following twelve days of public hearings in seven cities 
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throughout the State the Select Committee prepared a report 
containing seventeen recommendations {pp. 154-157). The initial 
concerns about discipline and public lack of confidence were 
not mentioned in the recommendations, but attention was given 
to Improvement of general curriculum, at least at the elementary 
tevel. The Committee expressed the belief that the number of 
subject areas taught in grades 1 through 6 should be reduced so 
that the teaching of basic $l<ills, especially reading, could be 
given more emphasis. The Committee recommended that reading 
be taught as a subject in grades 1 through 8. 
Other Committee recommendations included: 

(1) decreasing the pupil-teacher ratio to 25-1, especially (n 
grades 1 through 3, and calculating the ratio considering 
only those teachers carrying a classroom load. 

(2) financing elementary and secondary education at higher 
levels. 

(3) increasing teacher competence through specified changes 
in teacher preparation programs: 

a) requiring all elementary education majors to take at 
least one course in reading methods. 

b) increasing the quantity and quality of on-the-job ex- 
periences in the preparation program. 

c) decreasing the number of required theory &nd methods 
courses so that future teachers may concentrate on 
courses in their subject fields. 

d) requiring instructional faculty in schools of education 
to have considerable classroom teaching experience 
at the elementary or secondary level. 

(4) administrative reforms such as 

a) providing for accountability and evaluation of admin- 
istrators. 

b) evaluating teachers via standardized testing of all 
students at all grade levels. 

c) changing the basis of funding formulas from average 
daily attendance to average daily membership. 

d) Increasing the number of principals in small schools 
and the number of assistant principals in large schools. 

e) staffing the State Board of Education with adequate 
research and secretarial personnel. 

f) passage of a State law requiring that school board 
members be elected. 
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There were only three areas of concern identified in the 
present study which were not specifically mentioned In the report 
of the House Select Connmittee. These were 'inadequate facili- 
ties*, and 'vocational education' and 'special education programs'. 
Thus the two studies tended to validate each other with regard 
to identification of critical issues in Tennessee education in 
1973«74. The present study added objective evidence of the 
relative priorities of these issues as viewed by six segments of 
that portion of the State's population most directly concerned 
with the educational process. 

SPECIFIC GROUP AND SUB-GROUP DIFFERENCES 

The remarkable degree of agreement on the relative priorities 
of issues among the diverse groups sampled in the present study 
has been mentioned previously. Correlations between sets of 
rankings showed near-perfect agreement between the two groups 
of fiscal authorities (city council and county court members), 
between fiscal authorities and school board members, between 
principals and teachers. Rather good agreement (i.e., significant 
at ,05 level or better) was found between principals and teachers, 
and both fiscal authorities and school board members. Superin- 
tendents constituted the most divergent group, showing sub- 
stantial agreement only with school board members. 

Group intercorrelations provided conceptual validity for com- 
bining certain groups to form clusters. Some differences of 
opinion on issue priorities became more apparent when city 
council and county court groups were combined to form a cluster 
of 'fiscal authorities', and superintendents, principals, and teach- 
ers were combined to form a cluster of 'school professionals'. 
Comparing sets of issue rankings for these two clusters with the 
ranking produced by school board members — the 'policy 
makers' — revealed these differences: 

1) fiscal authorities were more concerned about teacher 
competence and discipline and relatively less troubled by 
the financing issue than were policy makers and school 
professionals. 

2) school professionals were more concerned about financing 
and si^e of classes than were members of the other clus- 
ters. 

3) school hoard members generally took a middle position on 
the issues — between fiscal authorities and school pro- 
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fessionats. On one issue, 'vocational education programs/ 
the policy makers assumed a more extreme position. They 
viewed vocational education as the most important issue of 
all, whereas fiscal authorities ranked it third and school 
professionals fourth. Undoubtedly school board members 
were preoccupied with vocational education because the 
tremendous increase in State funding of programs in this 
area has created a need for new directions in policy and 
capital outlay. 

Several of the groups of Tennesseans sampled in the present 
study could be sub-divided for further exploration of opinion 
within the groups. Comparison of the rankings of elected and 
appointed school board members showed a very high level of 
agreement between these two sub-groups, and between the two 
sub-groups and prevailing State-wide opinion as represented by 
the Tennessee composite ranking*. Both elected and appointed 
school board members did, however, tend to see financing as a 
less important issue than did the Tennessee sample in general. 
Predictably, school board members appointed by quarterly county 
courts and city councils showed a somewhat higher level of 
agreement with these fiscal authorities than did elected school 
board members. 

The superintendent grouping was sub-divided to distinguish 
differences in opinion by county superintendents elected by 
popular vote, county superintendents appointed by quarterly 
county courts, and city superintendents — all of whom were 
appointed by their local school boards. A significant degree of 
consensus on issue priorities was found to exist among the 
superintendent sub-groups, but only elected county superin- 
tendents showed substantial agreement with the Tennessee 
composite ranking. Appointed county and city superintendents 
were much less concerned about 'discipline' and more concerned 
about special education than was the Tennessee sample in 
general. The most divergent sub-group of all — city superin- 
tendents — viewed 'improvement of general curriculum' as a 
more important issue and class size as less important than other 
Tennesseans surveyed. All superintendents were more troubled 
by 'inadequate facilities* and {ess bothered by discipline problems 
than were the other groups. 

Interestingly enough, elected county superintendents were in 
better agreement than the appointed superintendents with county 
court members — responsible for appointing the ^appointed' 
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county superintendents — and with school board members — 
responsible for appointing all city superintendents. Apparently 
the elected superintendents, caught up in the political process, 
were much more finely aUuned to prevailing sentiment among 
their constituents than v. .?re the appointed superintendents. The 
appointees, especially city superintenderits whose school board 
appointments removed them farthest from politics, appeared to 
think more independently. They were less interested in the more 
interpersonal, short-range issues of discipline and apathy and 
concentrated more on the rather impersonal, broad, on-going 
goal of curriculum improvement, including special and vocational 
education. 

Principals were classified according to the organizational 
level of the school administered: 'elementary\ ^secondary', or 
^combined' (grades 1-12). There were too few responses from 
'combined' school principals to warrant inclusion of this category 
in statistical analyses. There was, in general, rather good agree- 
ment (significant at .05 level or better) between elementary and 
secondary principals on issue rankings, and between both sub- 
groups and the Tennessee composite. But some differences 
stood out: 

1) secondary principals agreed with the Tennessee com- 
^ posite ranking of 6 for the issue 'size of classes'. Elemen- 
tary principals viewed class sue as second only In 
importance to 'financing education', and with good reason. 
In a year when the national average pupil/teacher ratio 
was 20.2/1 (Wyngaard, 1974) the ratio in elementary 
schools in Tennessee was 29.1/1 (Tennessee State De- 
partment of Education, 1974). 

2) the issue 'vocational education programs' was given a 
much lower priority by elementary principals than by 
secondary principals and by the Tennessee sample In 
general. 

3) secondary principals were much more concerned about 
'improvemeni of general curriculum' and somewhat less 
concerned about 'discipline' than were elementary prin- 
cipals and all Tennesseans surveyed. 

Agreement between elementary teachers and their principals 
and between secondary teachers and their principals regarding 
priority issues was nearly perfect. Elementary teachers showed 
less concern about 'vocational education programs' and sub- 
stantially more concern about class size than either secondary 
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teachers or the Tennessee sample in general. The difference 
between elementary and secondary teachers on the issue of 
class size can be explained by these statistics: in 1972-73 whon 
the pupil/teacher ratio for secondary teachers in Tennessee vvas 
17.7/1, the ratio at the elementary level was 29.1/1 (Tennessee 
State Department of Education, 1974). ^ 

Like secondary principals, secondary teachers were more 
concerned about Improvement of general curriculum' than ele- 
mentary teachers and the Tennessee survey sample. Secondary 
teachers were less concerned than elementary teachers about 
the turbulent kind of problems associated with the 'discipline* 
issue, but they were much more troubled by the apathy implied 
in the issue 'lack of concern' than were either elementary teach- 
ers or the Tennessee sample in general. 

'Teacher competence', perhaps predictably, was not viewed 
by either teacher sub-group with quite the concern expressed by 
the ranking of 2 which this issue received in the Tennessee 
composite. 

The level of agreement between elementary and secondary 
teachers was lower than that between most other groups and 
sub-groups included in this survey, and \W\s was due largely to 
substantial differences on three issues: 'size of classes', 'im- 
provement of general curriculum', and Vocational education 
programs*. 

COMPARISONS BY SYSTEM TYPE AND GEOGRAPHIC AREA 

Further analysis of the survey data involved preparing sum- 
mary and composite rankings for ad groups in each of the fol- 
lowing categories: 

1) cpun^y and city/special district school systems 

2) East, Middle, and West Tennessee 

3) Tennessee's nine Planning and Development Regions 
The high level of agreement between various groups con- 
cerned about education in Tennessee was further substantiated 
by the analysis based on these three sets of categories. There 
was near perfect agreement on issue priorities among groups 
associated with county school systems, among groups associated 
with city/special district systems, and between the composite 
rankings for the two types of systems. 

Groups of Tennesseans surveyed in East, Middle, and West 
Tennessee agreed among themselves on the relative importance 
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of the survey issues within their own Grand Divisions. When 
composite rankings for the Divisions were compared, it was found 
that no differences existed between IVIiddle and West Tennessee. 
The level of agreement between groups in East Tennessee and 
those in the other two Divisions was substantial (significant at 
.001 level) but East Tennesseans were less concerned about 
^discipline' and saw 'lack of concern' and 'vocational education 
programs' as more important issues than Middle and West 
Tennesseans. 

When survey responses were categorized by Planning and 
Development Region statistical analyses revealed a high degree 
of consensus among the various groups within each of the nine 
Regions. Coefficients of correlation between composite rankings 
indicated that opinion of the State-wide sample in general regard- 
ing issue priorities was substantially mirrored by opinion in each 
of the nine Regions. 

RELIABILITY AND VALIDITY OF SURVEY INSTRUMENT 

The survey instrument used in the present study contained 
instructions for respondents to indicate the relative importance 
of the listed education issues in two different ways: ranking from 
1 to 10; and rating of A, B, or C (indicating, respectively, a rating 
'of critical importance', 'of some importance', or 'of little or no 
importance'.) Comparisonsf of the two sets of responses yielded 
a measure of "test-retest" reliability. The Spearman rank coeffi- 
cient of correlation between rankings and ratings exceeded .91. 

Validation of the list of survey issues as a listing of concerns 
that were of real importance to Tennesseans was made possible 
by an analysis of the 'A,' 'B,* and 'C ratings assigned to the 
issues by respondents. Approximately 80 percent of the survey 
participants considered all issues except 'administrative reform' 
to be of at least 'some' importance. The latter issue title appeared 
to be ambiguous — too vague to communicate the several rather 
controversial issues subsumed by the category — yet more than 
two-thirds of the survey respondents considered even this issue 
to be of at least 'some' importance. 

ADDITIONAL ISSUES SUGGESTED BY RESPONDENTS 

Only 12 percent of the survey respondents exercised the 
option to write in 'Other' concerns in the space provided on the 
survey instrument. Since approximately 80 percent of the written 
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responses could be classified as remarks related to the ten listed 
issues, the conclusion that the listing was a valid one was 
strengthened. 

No nev/ issue was suggested by more than four respondents. 
New concerns mentioned more than once included court-ordered 
busing, other problems related to pupil transportation such as 
overcrowded buses, need for more counselors at all levels and 
other specialists such as librarians and reading teachers, indi- 
vidualization of instruction, and teacher retirement plans and 
benefits. 

Most of the written remarks appearing on the survey instru- 
ment were related to four of the listed issues: 'financing educa- 
tion', 'teacher competence', 'improvement of general curriculum*, 
and 'administrative reform'. Finanjcing was viewed by many as 
an overarching issue: if sufficient funds were available, most of 
the other issues could be resolved. Two factors provided an 
indication that perhaps more funds for public education in Ten- 
nessee were becoming available. The House Select Committee 
(Tennessee General Assembly, 1973) recommended that more 
State money be channeled into elementary and secondary edu- 
cation. Analysis of usage of federal revenue-sharing funds re- 
vealed that in 1973 most Tennessee counties and municipalities 
put their money into local education programs. A State income 
tax was suggested as a means of raising more State revenues for 
education, and feelings expressed in this survey indicated that 
Tennesseans concerned about education might support an in- 
come tax proposal. 

The high ranking of 'teacher competence' and the quantity 
of written remarks related to this issue contributed to the con- 
clusion that Tennesseans in 1973-74 were seriously troubled by 
the performance of their teachers. Deep dissatisfaction with 
current teacher preparation programs at some colleges was 
expressed. Additional realism, provided by more on-the-job ex- 
perience in preparation programs, seemed to be a demand. 
Inability of teachers to properly individualize instruction and to 
teach basic skills (reading, writing, arithmetic) was a related 
concern. 

Teacher apathy, lack of dedication, lack of professionalism 
were mentioned as critical concerns by teachers as well as the 
other groups of Tennesseans surveyed. There was some feeling 
that present tenure regulations resulted in retention of incom- 
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petent, indifferent teachers. Improvements were suggested for 
in-service education programs and for teacher evaluation pro- 
cedures. 

Removal of politics from education, especially touching upon 
the hiring and firing of teachers and administrators, was a key 
element of the concern about 'administrative reform and/or 
reorganization'. It was suggested that popular election of school 
board members {rather than appointment by county court) and 
appointment of superintendents by boards of education might be 
steps in the right direction. Several individuals expressed the 
opinion that qualifications for school board members should be 
raised and that boards should be provided in-service education 
to enhance their effectiveness. 

Modest support was given by respondents to the idea of con- 
solidating all school systems within a county as an efficiency 
measure. Finally, teachers registered a strong plea, as they did 
in the legislative action survey sponsored by the Tennessee 
Education Association (1974), for release from such "adminis- 
trative" responsibilities as keeping of attendance records, lunch 
room patrol, supervision of school bus loading, and other extra- 
classroom responsibilities. Additional paraprofessional and/or 
clerical personnel would seemingly provide the kind of assistance 
the teachers have requested. 

LIMITATIONS OF THE SURVEY 

Opinion surveys generally are plagued by ambiguities, un- 
explained contradictions, and a host of other built-in limitations. 
While the validity and reliability of the survey instrument used in 
the present study apparently reached highly ^^cceptabfe levels, 
there were still issue titles such as 'administrative reform and/or 
reorganization' which did not fully communicate to respondents 
the author's perception of the given issue. 

Perhaps the fal! was not a good time to mail a questionnaire 
to the groups most concerned with education. Certainly the re- 
turn of 51 percent of the survey instruments was disappointingly 
low. The low response rate from city council, county court, and 
school board members was especially disconcerting. The medi- 
ocre returns were difficult to explain since virtually no negative 
feedback was received regarding either the survey instrument 
or the survey itself. 

Professional opinion was better represented in the survey 



52 



than lay opinion, though still at mocfest levels. Geographic areas 
of the State were well represented, but there was an imbalance 
in representation of school system types. City/special district 
schoo! systems were better represented than county systems. 
Also, the response from small cities was much greater than that 
for the four largest metropolitan areas. 

V. SUMMARY AND IMPLICATIONS 

SUMMARY 

In October 1973 representatives of six groups of Tennesseans 
reflecting professional and lay opinion about education were 
asked to Identify what they considered to be the most critical 
curreint issues in Tennessee education. The most frequently 
mentioned responses in this initial phase of the survey were used 
to construct a listing of ten key issues which was printed on a 
business reply card along with instructions for ranking the issues 
in order of importance. In a second phase the survey instrument 
thus developed was sent to a larger sample of the same six 
groups of Tennesseans most concerned about the educational 
process: city council, county court, and school board members 
representing the lay point of view; superintendents, principals, 
and teachers representing the professional position. Between 
November 1973 and the end of January 1974, fifty-one percent of 
the stratified random sample selected returned completed survey 
instruments. 

A remarkably high degree of association was found to exist 
among the opinions of the six groups of Tennesseans with regard 
to the relative importance of the survey issues. The ten critical 
Issues in Tennessee education in 1973-74, as identified and 
ranked by six professional and lay groups most directly con- 
cerned with education were: 

1. Financing education — including salaries 

2. Teacher competence 

3. Vocational education programs 

4. Discipline 

5. Lack of concern by pupils, staff, parents, and public 

6. Size of classes — overcrowding and overloaded staff 

7. Improvement of general curriculum 

8. Inadequate facilities 

9. Special education programs 

10. Administrative reform and/or reorganization 
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When responses were analyzed according to school system 
type and geographic area of the State represented, a highly 
significant level of agreement regarding the relative importance 
of the issues was found to exist among the surveyed groups of 
Tennesseans associated with (1) county school systems, (2) city/ 
special district school systems, (3) East Tennessee, (4) Middle 
Tennessee, (5) West Tennessee, and (6) each of the State's nine 
Planning and Development Regions. Good agreement on issue 
priorities was also found between combined group rankings for 
(1) county systems and city/special district systems, (2) East, 
Middle, and West Tennessee, and (3) the nine Planning and 
Development Regions. 

IMPLICATIONS 

While some differences in opinion regarding specific issue 
priorities were found between some sub-groups of the survey 
sample, notably between elected and appointed superintendents, 
elementary and secondary principals, and elementary and sec- 
ondary teachers; the most important finding was the remarkable 
degree of consensus among the Tennesseans sampled. One 
conclusion stood out very clearly: in 1973-74 there was a group 
of "critical issues in Tennessee education/' and an order within 
that grouping, upon which diverse groups of Tennesseans con- 
cerned about education could agree. This being the case, edu- 
cators, legislators, educational policy makers, and faculties en- 
gaged in teacher training throughout the State should take note 
of these priority issues and be guided by some of the associated 
implications, 

1. Financing education. According to current figures published 
by the National Education Association Tennessee ranks 46th in 
the nation in state spending per pupil ($730/pupil compared with 
the national average of $1034) and 43rd among the states In 
average teacher salary ($8450 compared with the national aver- 
age of $10,643) (Wyngaard, 1974). To insure that the youth of 
Tennessee are provided with facilities and educational opportuni- 
ties that make their achievement level comparable with that of 
youth in other states, Tennessee's per pupil expenditure and 
teachers' salaries should be brought closer to the national aver- 
age. To provide the extra funds needed for educational excellence 
new sources of State revenues must be tapped. Some of the 
survey respondents suggested a State income tax. Politicians at 
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the State level are understandably reluctant (o set in motion the 
machinery that could produce an income tax system. Several 
legislators have expressed the opinion that since education 
stands to gain the most from the new revenue source, educators 
and associated lay groups should spearhead the drive for an 
income tax. 

Tennesseans most directly associated with education agree 
that financing education is the key Issue in Tennessee education. 
A State income tax has been proposed as the best source of new 
revenue. Thus, in the interest of achieving educational excellence, 
organizations such as the Tennessee Education Association, 
School Boards Association, State Board of Education, Higher 
Education Commission, and the Parent Teacher Association, 
should band together to organize a Stale-wide program to inform 
the public of the need for an income tax. 

2. Teacher competence. First the widespread practice of 
assigning teachers to courses outside their areas of certified 
competence must be drastically curtailed. Instruction of high 
quality cannot be guaranteed when large numbers of teachers 
are required to teach courses for which they are not qualified 
in order to ''round out their schedules." 

Tennesseans* concern about teacher competence must be 
met squarely by the teacher training institutions of the State. 
Observers of the public schools see individuals filling positions 
as teachers who are not committed to education as a career and 
have neither the temperament nor the ability to respond to the 
needs of a classroom of children. Colleges of education need to 
improve (a) their methods of selecting candidates for teacher 
training and (b) career education for their own students. 

The shrinking demand for teachers in most areas which will 
probably continue in the coming years provides a favorable 
climate for reappraisal of admitting policies for teacher prepara- 
tion programs. Now, more than ever before, quality of candidates 
can be emphasized at the expense of quantity o' output. Combi- 
nations of personality inventories, interviews, and various assess- 
ments of background experiences should be employed to screen 
applicants for teacher training in order to assure that those 
accepted are mature individuals who truly enjoy working with 
youngsters .'^nd can handle this responsibility effectively. Once 
accepted, the teacher-in-training must be given more opportuni- 
ties than he now receives in many institutions to observe, and 
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to participate in, actual teaching situations so that his choice of 
teaching as a career can be confirmed (or perhaps rejected) on 
the basis of realistic first-hand information. 

Some observers who participated in the present survey felt 
that standards for evaluation of teacher trainees' performance in 
course work and the teaching internship varied significantly 
among the training institutions in the State. Perhaps school 
systems need to employ their own performance evaluations in 
hiring new teachers. Screening of an applicant might include 
observation of his performance in a simulated classroom situa- 
tion. Another method of assuring quality among new teachers 
might be the initiation of "new-teacher institutes" in each school 
system. The purposes of such institutes might include (a) allowing 
new teachers to discuss frankly their job-related problems with 
each other and with more experienced teachers or supervisors 
who could suggest possible solutions, (b) acquainting the new 
teachers with the system's resources: materials, equipment, and 
personnel, and (c) providinv^ the school system w\h data on 
which to make the decision to re-hire or dismiss a teacher at 
the end of his first ye^*" on tht> job. 

Teacher training institutions also need to reassess their ef- 
forts in the following areas to see what improvements might 
be made: 

(a) preparing elementary teachers to teach reading skills and 
diagnose learning difficulties in this area of development 

(b) preparing teachers at ail levels to individualize instruction 

(c) providing models and resources for in-service education 
of teachers and administrators 

(d) suggesting standards and procedures for performance 
evaluation of practicing teachers and administratcrc. 

3. Vocational edtication programs. Practicing professionals at 
all levels of education — elementary, secondary, teacher training 
— need to improve what they are doing as part of their own 
courses to provide their students with realistic information about 
possible careers and criteria for making appropriate career 
choices. At junior high, senior high, and college levels students 
should be provided with marketable skills in their chosen occu- 
pational areas. 

4, 5, Discipline and lack of concern by pupils, staff, parents, 
and public. Lack of concern is not an issue just in Tennessee. 
The loss of pubfic confidence in educational institutions on a 
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national level is well documented. The 1974 Carnegie Commission 
Report notes that education has been beset by a series of crises 
in the last few years — student unrest, political reaction, financial 
distress, and now a crisis of confidence (Mathews, 1974). This 
crisis of confidence is apparently part of the crisis of pessimism 
carrenlly pervading the country. With mistrust and apathy on the 
part of the public making their jobs harder, many teachers seem 
to have become apathetic too. Students have lost their respect 
for authority, and the result is often behavior that results in disci- 
pline problems for teachers and for schools. 

Public confidence in its social institutions is not likely to be 
restored quickly or easily, but restoration of confidence in edu- 
cation could be furthered significantly by the attention to critical 
issues herein recommended. In addition, school systems should 
employ more effective public relations techniques to keep the 
put^lic informed of innovations and on-going programs. 

6. Size of classes — overcrowding and overloaded staff. 
Much disagreement exists concerning the proper means of cal- 
cufating pupil/teacher ratio. Different groups use various figures 
to come up with a ratio that serves their purposes. However, 
there is virtually no disagreement with the conclusion that the 
pupil/teacher ratio needs to be lowered in Tennessee, especially 
at the elementary level. 

Agreement on the method of calculating pupil/teacher ratio 
needs to be reached. State guide-lines for maximum teacher load 
need to be enforced more strictly. Sun/ey participants deplored 
the readiness with which teacher overloads are approved. In- 
creasing the number of teacher aides State-wide would also help 
to alleviate the problems associated with large classes. These 
and other methods of reducing the pupil/teacher ratio in Ten- 
nessee should receive a high priority among the issues on which 
the Tennessee Education Association seeks action. 

7. Improvement of genera! currSculum. Public confidence in 
today's edutational processes was further shaken by the recent 
announcement by the College Entrance Examination Board that 
scores on their Scholastic Aptitude Test have declined during 
the last decade. It has been hypothesized that under the influence 
of the electronic media children see and hear more but read 
less and think less deeply than preceding generations {Hechinger, 
1974). Individuals responding to the present survey expressed 
deep dissatisfaction with schooling that is less analytical and 
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less print-oriented than heretofore. They called for more emphasis 
on the basic skills — reading, writing, arithmetic — in the early 
grades so that remedial work at the high school and college 
levels would not be necessary. 

In light of current criticisms and the decline in test scores, 
perhaps it Is time for system-wide reassessments to see if im- 
provement of the general curriculum is warranted. Are the 
schools really geared for accomplishment of the broad goals 
society has set for its educational institutions? 

Perhaps the importance of reading as a tool for understanding 
any subject is not receiving the appropriate emphasis. Perhaps 
spelling, cursive skills, and diction are passed over lightly in the 
evaluation of students' work. Perhaps the methods for teaching 
reading, writing, and arithmetic that are being imparted to train- 
ees in teacher preparation programs simply are not very effective. 
Whatever the reasons, the means for achieving some long-estab- 
lished curriculum goals seem to be in question. 

8. Inadequate facilities. A previous study by the author (Banta, 
1973) revealed that in 1972-73 more than one/fourth (26.8 percent) 
of all Tennessee schools represented by response to a State-w!de 
survey were enrolling more students than the school plant was 
designed to sen/e adequately (p. 285). Construction of new 
facilities was viewed as a critical need by 37 percent of the prin- 
cipals responding. Remodeling and improved maintenance of 
existing facilities were cited as the needs of an even larger 
proportion of the principals. 

More space is needed throughout the State for libraries, 
indoor play areas, and vocational classes. Many additions to 
existing school campuses are required just to relieve overcrowd- 
ing in regular classrooms. Where the student body of an over- 
crowded school exceeds 1000, one or more new schools should 
be built and the students divided between them to ensure the 
best possible educational program for aff concerned. Extending 
the school year to include twelve months of operation is another 
method ot aUev^aling overcrowding that would not require con- 
struction of as many new buildings, Better maintenance of present 
facilities is a desperate need, according to the principals partici- 
pating in the 1973 study. 

9. Special education programs. Effective implementation of 
current legislation should result in improved special education 
for the gifted and the handicapped. More facilities are needed to 
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accommodate increased programming. And teacher training In- 
stitutions must adapt to meet the demand for more special educa- 
tion teachers if ait handicapped youth in the State are to be given 
an opportunity to attend a public school. 

10. Administrative reform and/or reorganization. Education 
must be separated as much as possible from the influence of 
politics. School boards should probably be elected, but qualifi- 
cations for candidates should be established at the local level to 
assure that school board members will possess the proper edu- 
cation, maturity, and related background experiences to carry 
out their responsibilities as policy makers in a competent manner. 
In-service training programs for school board members could 
help to assure a creditable performance by this influential group. 
Superintendents should probably be appointed by school boards 
because, as was indicated in the present study, this apparently 
places the superintendent far enough from the influence of 
politics that he is able to exercise his professional judgment with 
considerable independence. On the other hand, the hiring and 
firing of teachers should be a responsibility of professionals in 
the field, not of the lay policy makers. 

In the interest of administrative efficiency and educational 
excellence political concerns should be set aside in some coun- 
ties to permit consolidation of very small school systems within 
the county. Some obsen/ers even advocate a single system for 
every county. 

The number of teachers required for appointment of a prin- 
cipal should be lowered in order to reduce the number of greatly 
overburdened teacher-principals in very small schools. 

Many teachers seek relief from what they consider onerous 
non-teaching responsibilities, e.g., patrolling the lunchroom or 
school bus loading, and keeping detailed attendance records. 
School administrators must provide supplementary clerical staff 
and adequate numbers of paraprofessionals to free the teacher 
to make the professional contribution for which he was trained. 
Sufficient staffing to assure each teacher one free period per 
day for planning is imperative. 

SUGGESTIONS FOR ADDITIONAL RESEARCH 

The individuals and groups that influence educational policy 
in Tennessee need the kind of information which could be pro- 
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vided by periodic updating of the present study. However, a 
measure of public opinion should be added to complete the 
picture of Stale-wide concern about issues in education. 

Before education forces can unite to inform Tennesseans of 
the need for an income tax there must be additional polling of 
teachers and administrators throughout the State to determine 
the extent of grass-roots fsupport for such a movement. 

Further probing of the wide-spread dissatisfaction with 
teacher preparation programs is needed. It was not clear from 
response to the present investigation whether criticism was being 
(eveted at at( the State's programs, or only at selected ones. 
Another study should be designed to identify specific targets of 
criticism and to gather the suggestions of practicing educators 
for improvement of preparation programs. 

Since removing political influence from the operation of 
educational institutions was an important source of concern 
expressed in the present study, one or more long-range evalua- 
tion studies should be initiated to determine the relative merits 
of electing and appointing school board members and superin- 
tendents. Does the school board member who answers directly 
to ihe voters of his district feel freer to follow his own course 
than one whose appointment was made by a city council or 
quarterly county court? Does the elected superintendent tend to 
suggest the try-out of new policies and programs more readily 
than one who is appointed? More importantly^ does the public 
really want a school board or a superintendent that has freedom 
to experiment with innovations; or would the people prefer that 
attention be focused primarily on those issues they consider of 
most importance at any given time? Such questions cannot be 
easily answered, but an intensive longitudinal investigation would 
certainly provide information of sufficient importance to justify 
the time and money it would require. 

The question of consolidation of the school systems within 
each county certainly deserves, and in some cases is getting, 
extensive evaluation. 
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PHASE ONE INSTrtUMENT AND LETTERS 



THE UNiV£F)Sinr OF TENKICSSCE 

COC^tOl or COwCHTiON 
KNOXViLLE. TENNESSEE 379t« 



iUAUU Of t&UCAT)OMi f^tSUftCH AND ttttVKi 



October 1, 1973 



Dear SuperJntertdent: 

The Bureau of Educational Research and Service at the University of 
Tennessee, KnoxvlUe would like to find out what thfnk are the crftlcal 
issues tn education In Tennessee today. 

The Bureau Is sampling the opinion of various professional and lay groups 
wUb a direct responsibility for education In Tennessee so that the State 
legislature, colleges of education, and local education agencies may become 
more responsive to the key Issues and problems in education as viewed by these 
groups. 

This study Is being carried out in two phases. Pirst, a small sample of 
superintendents and one principal, one teacher, one school board member, and 
one county court or city council (whichever body has fiscal responsibility for 
schools in a given systeii) member in each of the State's 147 school systems Is 
being contacted for input regarding their perceptions of crucial issues. (This 
is the phase in which we are seeking your cooperation.) Responses to the first 
mailing will be analyzed and a set of five to seven issues Y*Mch appear to be 
of most concern will be complied* 

In the second phase of the study this list of specified issues will be 
sent to all superintendents and a ten percent random sample of principals, 
teachers, school board meir^bers, and county court or city council members In 
each school system* These individuals will be asked to rank the specified 
issues in order of Importance as they see them. Comparisons Ci(\ th^n be mide 
between the rankings of critical €dt;cat(onal issues by each of these groups 
of concerned individuals. 

Please take a few minutes to complete the enclosed self-addressed card 

and send us your views. Your input is especially important because at this 

stage of the investigation you are one of just fifteen superintendents in 
the State being contacted! 

We look forward to receiving your reply very soon. Please try to mail 
the enclosed card before October 20 . Thank you very much for your assistance. 



Sincerely, 




Trudy w. Banta 
Special Project Director 



TWB:ces 
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THt UNlVCRSiTY OF.TCNNE&SCC 

COtlEOI Of tCHKATiOH 
KNO)CViaC TENNESSEE 37916 



mtA\J Of lOUCATlONAL flUCAPKH AND ilfWiCt 



October 12» 1973 



DeAr City Council Moobert 

Ttie Bureau of Education Reiearch and Service at the University of Tennessee, 
XnoxvUle vould like to find out what you think are the critical issues in edu- 
cation in Tennessee today« 

the Bureau is sanpllng the opinion of various professional and lay groups 
vlth a direct responilblltty for education In Tennessee so that the State legis- 
lature, colleges of ciucatloni and local education agericles nay becoae more respon- 
sive to the key issues and problons In education as viewed by these groups. 

This study Is being carried out In tvo phases. First, a small sanpte of 
superintendents, and one principal, one tescher^ one school board mepber, and one 
county court or city council (whichever body has fiscal responsibility for schools 
In the given systen) raeober In each of the State's 147 school systems Is being 
contacted foe input regarding their perceptions of crucial Issues, (this is the 
phaie In which we are seeking your cooperation,) Responses to the first mailing 
vlH be analyzed and a set of five to seven Issues which appear to be of laost 
concern will be coapUed, 

In the second phase of the stu<jy this list of specified Issues will be sent 
to all superintendents and a ten percent randoca sample of principals, teachers, 
school board menbers, and county court or city council neabers associated vlth 
each school systen, these Individuals will be asked to rank the specified Issues 
in order of Importance as they See thcsn. Coaparlsons can then be toade between 
the rankings of critical educational Issues by each of these groups of concerned 
Individuals. 

Please take a few minutes to complete the enclosed self -addressed card and 
send us your views. Your Input is especially ixQportant because at this stage of 
the Investigation you are the only city council men^ber associated with your 
particular school systen being contacted. And In our study we are considering 
your opinion ss an elected official to be representative of the opinion of the 
voting public. On the reply card please provide the name of the school aystea 
(or systens) for which you as a city council mcniber have direct responsibility 
(we assune this responsibility la primarily fiscal). 

We look forward to receiving your reply very soon. Please try to mail tht 
cncloaed card before ^^ipber^gSi Thank you very much for your aaslstance. 




Sincerely, 



Special Project Director 



Hi 
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THCUNIVCRSlTVOF TENNESSEE 

COLLCCCOr tOUCAriON 
KNOXViLLE, TENNESSEE 37»l« 



■ UMCAU Of tOUCATiOHAL ftlSUlKH ANO Sl^lCC 



October 24, 1973 



Dear Teacher: 

The Bureau of Educatloaal Research and Service at the University ti 
Teitnessee, KnoKville would like to find out what you think are the critical 
Issues 1ft educatlofi In Tennessee today. 

The Bureau Is sampling the opinion of various professional and lay groups 
vlth a direct responsibility for education In Tennessee so that the State 
legislature, collegcsof education, and local education agencies may becone 
Doce responsive to the key issues and problems In education a^ vleved by these 
groups. 

This study Is being carried out In two phases. First, a saall sample of 
superintendents and one principal, ot\e teacher, one school board member, and one 
county court or city council (whichever body has fiscal responsibility for 
schools In a given systen) member In each of the Staters 147 school systems Is 
being contacted for Input regarding their perceptions of crucial issues* (This 
is the phase In which we are seeking your cooperation.) Responses to the first 
mailing will be analyzed snd a set of five to seven Issues which appear to be 
of most concern will be compiled. 

la the second phase of the study this list of specified issues will be 
sent to all superintendents and a ten percent random sample of principals, 
teachers, school board members, and county court or city council members In 
each school system, these Individuals will be asked to rank the specified 
Issues In order of Importance as they see them. Comparisons can then be made 
between the rankings of critical educational Issues by each of these groups of 
concerned Individuals. 

Please take a few minutes to complete the enclosed self'^addressed card and 
send us your views. Your Input is especially Important because at this stage 
of the Investigation you are the only teacher In your system being contacted. 

We look forward to receiving your reply very soon. Please try to oall the 
enclosed card before October 31 , Thank you very much for your asslstan:e. 



Sincerely, 




Trudy w. Banta 
Special Project Director 
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PHASE TWO 
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THE UNIVtRSlTVOr TENNC&SEE 

tCUlGtOr lOUCATtOW 
KNOXVILLC, TENNESSEE 37916 



BUWUU Of (OoCAtiOML RtSt>f»CH ASO SCRVlCl 



Kovember 13, 1973 



Dear County Court Member: 

The Bureau of Educational Research and Service at the Univerjity of Tennessee* 
KnoxvlUo would like to know hov various groups of Tennesseans view the critical 
Issues tn Tennessee cdvicitlon today. Such information should be useful In naklng the 
State legislature) colleges of education, and local education agencies more respon- 
sive to the key issues and problems in education as Identified by these groups* 

tXirtng October, superintendents, principals, teachers, school boar<l members, and 
county court or city council rnenbers representing each of Tennessee's 146 school 
syatans were asked to identify crucial issues in Tennessee education. From their res* 
ponses a list of the ten most frequently mentioned issues has been compiled. As a 
representative of one of these groups of Tennesseans concerned about education, we 
would like to have your opinion rcgacdln& the relative Importance of these Issues. 

Please take just a moment now to consider the ten Issues listed on the enclosed 
reply card* the issvies are purposely stated in a broad, general manner In order to 
Include all sides of any controversy that my exist concerning a given issue* 

first, we woMld like to know how strongly you feel about thfc ten issues listed* 
You may feel that only two of the Issues are of critical importance and that the others 
deserve little attention. On the other hand, you may feet that all ten Issues are 
critical. The column of boxes on the reply card permits you to express yout petsonal 
degree of concern about the listed Issues. 

Secondly, please use the column of circles on the reply car 4 to xatiV. the teit 
issues in order of Importance as you see then* Assign the number 1, to the Issue you 
consider most important in Tennessee education today, and continue ranking the Issues 
until the number 10 Is assigned to the issue you hclinve t» be least important. 

Since you taay feel that a critical issue has been oraltted, space has been provided 
for you to list *other* concerns you may have. 

Ve appreciate your assistance* Please return the self "Addressed reply card by 
ypvcrober 21. 



Sincerely, 



Trudy W> Banta 

Special Project Director 
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NCAU 0# IOOCATONAL RUCAACH AHO SClVVICf 



THC UNIVERSITY or TtNNESSCC 

COllCCI Of IDUCAT»ON 
KN0XVILL£^TCNNCSS£E 37dt6 



Novwaber 16, 1973 



Dear School Board Member; 

The Bureau of Educational Research and Service at the University of Tennessee, 
Knoxvllle would like to know how various groups of Tennesseans view the critical 
Issues In Tennessee eJucaticn today. Such information should be useful In "making the 
State legislature, colleges of education, and local education agencies more respcn* 
•ive to the key issues and problems lt\ education as Identified by these groups. 

During October, superintendents, principals, teachers, school board members, and 
county court or city council nenbers representing each of Tennessee's 1^6 school 
systcris were asked to identify crucial Issues in Tennessee education. From their res- 
ponses a list of the ten most frequently mentioned issues has been compiled. As a 
representative of one of these groups of Tcnnesseans concerned about education, ve 
would like to have your opinion regarding the relative importance of these Issues. 

Please take just a monent now to consider the ten issues listed on the enclosed 
reply card. The Issues are purposely stated In a broad » general manner In order to 
Include all sides of any controversy that may exist concerning a given Issue. 

First, we would like to know how strongly you feel about the ten issues listed. 
You may feel that only two of the Issues are of critical importance and that the others 
deserve little attention. On the other hand, you may feel that all ten Issues are 
critical, the column of boxes on the reply card permits you to express your personal 
degree of concern about the listed Issues. 

Secondly, please use the coluinn of circles on the reply card to rank the ten 
Issues in order of Importance as you see then. Assign the number 1^ to the Issue you 
consider most Important in Tennessee education today, and continue ranking the Issues 
until the number 10 Is assigned to the Issue yow believe to be least Important. 

Since you may feel that a critical Issue has been omitted, space has been provided 
for you to list 'other' concerns you may have. 

We appreciate your assistance. Please return the self -addressed reply card by 
Kovgnbcr 28 > 



Sincerely, 




Trudy W, Banta 

Special Project Director 
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THE UNIVtaSITYOr TENNESSEE 



»U«MU or ICHJCATK5ML RCitAJ^CH ANO S(RV»CC 



Kovember 16, 1973 



Dear Principal i 

Tho Bureau of Educational Research and Service at the University of Tennessee* 
Knoxvllle would like to kt\ow how vatrlous gtou^is of teuaesseans vlev» the critical 
Issuea In Tennessee education today. Such Information should bo useful In waking tha 
State legislature, colleges of education, and local education agencies more tespon- 
slve to the key Issues and problems In education as Identified by these groups. 

During October, stiperlntendents, principals, teachers, school board meiibers, and 
county court or city council ncmbers representing each of Tennessee's IA6 school 
aysteDS were asked to Identify crucial Issues In Tennessee education. From their res« 
ponses a list of the ten most frequently mentioned Issues has been compiled. As a 
representative of one of these groups of Tennesseans concerned about education, ve 
would like to have your Opinion regarding the relative Importance of these issues, 

^leasft takfe juat a trioneut riow to ronslder the ten Issues listed on the enclosed 
reply card. The issues are purposely stated In a broad, general manner In order to 
Include all sides of any controversy that inay exist concerning a given issue. 

First, ve would like to know how strongly you feel about the ten Issues listed. 
You may feel that only two of the Issues are of critical Inportance and that the others 
deserve little attention. On the other hand, you may feel that all ten Issues are 
critical. The column of boxes on the reply card perrftlts you to express ycur personal 
degree of concern about the listed* Issues* 

Secondly, please use the columrt of circles on the reply card to rank the ten 
issues in order of importance as you see thm* Assign the number 1^ to the issue you 
consider most Important in Tennessee education today, and contlnue^ranking the Issues 
until the number 10 is assigned to the issue you believe to be least important. 

Since you oay feel that a critical issue has been omitted, space has been provided 
for you to list 'other* concerns you may have* 

We appreciate your assistance* Flease return the self ^addressed reply card by 
Wovenbet 28 . • 



Sincerely, 




Trudy W. Banta 

Special Proiect Director 
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OP TENNESSEE 

COUNTIES IN GRAND DIVISIONS 

West 



Benton 

Carroll 

Chester 

Crockett 

Decatur 

Dyer 

Fayette 



Bedford 

Cannon 

Cheatham 

Clay 

Coffee 

Davidson 

Dekalb 

Dickson 

Fentress 

Franklin 

Giles 

Grundy 

Hickman 

Houston 



Anderson 

Bledsoe 

Blount 

Bradley 

Campbell 

Carter 

Claiborne 

Cocke 

Cumberland 

Grainger 

Greene 



Gibson 

Hardeman 

Hardin 

Haywood 

Henderson 

Henry 

Lake 

Middle 

Humphreys 

Jackson 

Lawrence 

Lewis 

Lincoln 

Macon 

Marshall 

Maury 

Moore 

Montgomery 

Overton 

Pickett 

Putnam 

East 

Hamblen 

Hamilton 

Hancock 

Hawkins 

Jefferson 

Johnson 

Knox 

Loudon 

McMinn 

Marion 

Meigs 



Lauderdale 

McNairy 

Madison 

Obion 

Perry 

Shelby 

Tipton 

Weakley 



Robertson 

Rutherford 

Sequatchie 

Smith 

Stewart 

Sumner 

Trousdale 

Van Buren 

Warren 

Wayne 

White 

Wiliiamson 

Wilson 



Monroe 

Morgan 

Polk 

Rhea 

Roane 

Scott 

Sevier 

Sullivan 

Unicot 

Union 

Washington 
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BY 
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TABLE A.I 

SUWMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES FOR FOUR GROUPS 
IN THE FIRST TENNESSEE PLANNING AND DEVELOPMENT REGION 





c<rv 

COUNC 


CNTY 
COURT 


SCH 
BRO 


SUPT 


PRINC 


isl TN 
TCHR COMPOSITE 


TCHR COMPETENCE 






1 


2 


3 


2 


2 


VOCATIONAL EOUC 


"O 

<5 


"O 
9 


2.5 


3 5 


2 


5 


3 


CLASS Size 


o 
ca 


O 
CD 


6 


9 


7,6 


3 


e 


OE^ CURRICULUM 


8 


O 

o 


5 


3.5 


55 


6 


5 






JZ 












SPECIAL EOUC 


U 
V> 


o 
to 


8 


6.S 


9 


10 


9 


FINANCING EOUC 


with 


r 

i 


2 5 


1 


} 


1 


1 


DISCIPLINE 


XJ 

o 


•o 
o 


7 


8 


75 


7 


8 




c 


e 












APATHY 


£ 


E 


4 


$ 


4 


4 


4 


FACILITIES 


o 
O 


o 
O 


9 


6.S 


5.5 


d 


7 


ADMIN REFORM 






10 


10 


10 


9 


10 


TOTAL NUMBER 
IN EACH GROUP 


(Da 


(2)a 


20 


6 


13 


38 


79 



a Number In parenthesis Indicates number of respondents from this 

group added to school board group to make the totai ol 20. 
Coefficient ot concordanca : .853 

Chi square, 9df : 30.696 

Probability of independenco : .000 



TABLE A-2 

SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES FOR SIX GROUPS 
IN THE EAST TENNESSEE PLANNING AND DEVELOPMENT REGION 





CITY 
COUNC 


CNTY 
COURT 


SCH 
BRD 


SUPT 


PRINC 


E TN 

TCHR COMPOSITE 


TCHR COMPETENCE 


1 


3 


3 


A 


3 


4 




VOCATIONAL EOUC 


2 


2 


1 


2.5 


7 


6 


3 


CLASS SIZE 


5 


7 


5 


7 


2 


2 


4 


GEN CURRICULUM 


7 


4 


9 


5 


4 


9 


7 


SPECIAL EOUC 


9 5 


10 


8 


6 


65 


6 


9 


FINANCING EDUC 


4 


f 


< 


1 


1 


1 


1 


DISCIPLINE 


6 


55 


4 


8 


55 


5 


6 


APATHY 


3 


5 5 


7 


9 


55 


3 


5 


FACILITIES 


8 


8 


e 


25 


'8 5 


7 


6 


ADMIN REFORM 


9 5 


9 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH CROUP 


9 


U 


19 


20 


21 


67 


150 



Coefficient of concordance : 660 

Chi square, 9df : 35.625 

PfObabitity of lnd«pef\dence : .000 




TABLE A-3 

SUMMARY AND COMPOSITE RANKINGS Of TEN EDUCATION ISSUES fOR FIVE GROUPS 
JN THE SOUTHEAST TENNESSEE PLANNING AN^ OEVELOPMENT REGION 





CITY 
COUNC 


CNTY 

COURT 


SCK 
BRD 


SUPT 


PRINC 


S E TN 
TCHR COMPOSITE 


TCH« COMPETENCE 




3 


3 


25 




6 


3 


VOCATIONAL EOUC 


t 


1 


1 


25 


a 


4 


1 


CLASS SIZE 


o 


6 


4.5 


5 


7 


1 


5 


















GcN CURHiCULUM 


3 


4 


7 


7 


3 


6 


7 


SPECIAL EDUC 


O 

o 


5 


6 






7 


6 


FINANCING EDUC 


c 

i 


9 


2 


1 


1 


2S 


2 


DISCIPLINE 


■o 
«> 
e 


2 


4 5 


9 


4 


2 S 


4 


APATHY 


1 


7 


8 


4 


5 


5 


6 


FACILITIES 


U 


a 


9 


8 


6 


9 


9 


ADMIN REFORM 




10 


10 


10 


10 


10 


10 


TOTAL NUMBER 
JN EACH GROUP 






(Ija 


7 


18 


16 


16 


32 


69 



a Number of city council members added lo county court group 

to m*)K9 Ihe lota) ol 7, 
Coefficient ol concordance .623 
CM square, 9df 28.018 
Probability ol independence .001 



TABLE A-4 

SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES FOR FIVE GROUPS 
IN THE UPPER CUMBERLAND PLANNING AND DEVELOPMENT FiEQiON 





CITY 
COUNC 


CNTY 
COURT 


SCK 
BRO 


SUPT 


PRINC 


TCHR 


UPR CUMB 
COMPOSITE 


TCHR COMPETENCE 




6 


6 


2 


7 


6 


5 


VOCATIONAL EDUC 




6 


4 


3 


9 


4.5 


7 


CLASS SIZE 




6 


2 


7 


2S 


1 


2 


GEN CURRICULUM 




^0 


9 


5 


6 


7 


8 


SPECIAL EDUC 




9 


75 


e 


8 


9 


9 


FINANCING EDUC 




3 


7 5 


1 


1 


4.5 




DISCIPLINE 




1 


3 


8 


4 


3 


3 


APATHY 




2 


1 


9 


S 


2 


4 


FACILITIES 




B 


5 


4 


2.5 


8 


6 


ADMIN REFORM 




4 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


0 


9 


8 


9 


5 


15 


48 



Coefficient of concord .6 : .412 

Chi square, 9df 18 561 

Probabihry of independence : 029 
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TABLE A-5 

SUMMARY AND COMPOSITE RANKJNGS OF lEN EOL/CATlON {SSOES FOR FiVE GROUPS 
$N THE WID-CUMBERLANO PLANNING AND DEVELOPMENT REGION 





1 T 

COUNC 


UNIT 

COURT 


SCH 

BRO 


SUPT 


PRJNC 


TCHR 


COMPOSITE 


TCHR COMPETENCE 




5 




2 


1 


6 


2 


VOCATIONAL EOUC 


t; 


1 


1 


4 


7 


a 


4 


CLASS SIZE 


O 

u 


7 5 


3 5 


9 


4 


3 


5 


GEN CURRICULUM 


2^ 
c 

3 


7 5 


6 5 


7 


5 


7 


7 


SPECIAL EOUC 


0 

o 


10 


6,5 


8 


6 


5 


9 


FINANCING EOUC 


i 


3 


5 


1 


2 


2 


1 


DISCIPLINE 




4 


2 


6 


3 


1 


3 


APATHY 


e 


2 


d 


3 


9 


4 


6 


FACILITIES 


1 


9 


9 


5 


6 


9 


a 


AOMIN REFORM 




6 


10 


10 


10 


10 


10 


TOTAL NUMBER 
JN EACH GROUP 


(l)a 


10 


9 


12 


17 


65 


113 



a Number of cAy council members combined with counly court 

group to make the to^a! of 10, 

Coefficient of concordance : ,526 

Chi square, 9dJ 23,6SO 

Probabifity ot mdependence : ,005 



TABLE A-6 

SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES FOR FIVE GROUPS 
IN THE SOUTH C£NTRAL TENNESSEE PLANNING AND DEVELOPMENT REGION 





CITY 
COUNC 


CNTY 
COURT 


SCH 
BRD 


SUPT 


PRINO 


TCHR 


S C TN 
COMPOSITE 


TCHR COMPETENCE 




1 


2 


2 


7 


6.5 


4 


VOCATIONAL EDUC 




4 


1 


1 


2 


6.5 


1 


CLASS SIZE 




9 


6 


9 


8 


1 


7 


GEN CURRICULUM 




3 


S 


a 


3 


5 


5 


SPECIAL EOUC 




10 


9 


5 


4 


9 


a 


FINANCING EDUC 




5 


4 


3 


1 


4 


3 


DISCIPLINE 




2 


7 


4 


5 


2 


2 


APATHY 




S 


S 


7 


6 


3 


6 


FACILITIES 




6 


3 


6 


g 


8 


9 


AOMIN REFORM 




7 


^0 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


0 


e 


13 


15 


9 


25 


70 



Coefficient of concordance i .453 

Chi square. Sdf 20.840 
Probability Of independence ,013 
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SUMMARY AND COMPOSITE RANKINGS OF TE^ EDUCATION ISSUES ^^OR FIVE GROUPS 
IN THE NORTHWEST TENNESSEE PUNNING AND DEVELOPMENT REGION 





CilY 
COUNC 


CNTY 
COURT 


SCH 

&RD 


SUPT 


PBtbiC 


TCHR 


N W TN 
COMPOSITE 


TCHR COMPETENCE 


2 


3 


1 


5 




3 


3 


VOCATIONAL EOUC 


3 


1 


a 


2 




6.5 


2 


CLASS SIZE 


5 


45 


6 


7 




2 


S 


QEN CURRICULUM 




8 


7 


9 


(« 
<b 


6.5 


9 


SPECtAL EOUC 


8 


6 


8 


6 


>- 

jC 


8 


8 


FINANCING EOUC 


1 


7 


3 


1 




1 


1 


0»SC*PLINE 


4 


2 


4 


8 


o 
c 


9 


4 


APATHY 


; 


4 5 


5 


4 


E 

o 


4 


0 


FACILITIES 


6 


9 


9 


3 


o 


5 




ADMiN REFORM 


10 


10 


10 


10 




10 


10 


TOTAt NUMBER 
IN EACH GF^CUP 


12 


10 


22 


20 


(4ja 


29 


93 



a Number of pnncfpaii combined wUh teacft«f gfoup to make th« 
lotal ot 29. 

Coetticiorti of concordance .643 
Chi square. 9df 28.925 
Probability of independence .001 



TABlE A-S 

SUMMARY AND COMPOSITE RANKINGS OF TEN EDUCATION ISSUES FOR FOUR GROUPS 
IN THE SOUTHWEST TENNESSEE PLANNING ANO CEVELOPWENT REGION 





CITY 

COUNC 


CNTY 
COURT 


SCH 
BRO 


SUPT 


PRINC 


TCHR 


S W TN 
COMPOSiTE 


TCHR COMPETENCE 




3 


3 




5 


4 


2 


VOCATIONAL EDUC 




e 


1 


■o 
« 


7 


1.5 


3 


CLASS sue 


o 
o 

>• 


4 5 


4 5 


o 

C3 


3 


3 


4 


GEN CURRICULUM 


c 

3 


7 


4 5 


hool 


6 


a 


7 


SPECIAL EDUC 


o 
O 


8 


7 


(J 


10 


7 


9 


FINANCING EOUC 




45 


2 


with 


1 


1.5 


1 


01SC<PHNE 


C 


t 


to 


« 


2 


s 


5 


APATHY 


E 


2 


7 


E 


6 


6 


6 


FAClUTtES 


o 
o 


9 


? 


o 

o 


4 


9 


6 


AOMIN REFORM 




to 


9 




9 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


dia 


7 


5 


(2)b 


5 


14 


3) 



a Number of cify council mernbers cofnbined with county court 

group to maVe the total ot 7. 
b Number of fiuperinfendenfs combined wiJn schooJ toa/d group 

!0 make the total of 5. 
Coefticienl of concordance .583 
Chi square. 9df 20 991 

Probability ot independence : ,013 
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TABLE 



SUMMARY AND COMPOSITE RANKfNGS OF TEN EDUCATION ISSUES FOR FOUR GROUPS 
IN THE MEMPHIS DELTA PLANNING AND DEVELOPMENT REGION 





CITY 


CNTY 


SCH 








MEM DELTA 




COUNC 


COURT 


8R0 


SUPT 


PRINC 


TCHR 


COMPOSITE 


TCHR COMPETENCE 




3 




1 


4 


S 


2 


VOCATIONAL EOUC 


r 


65 


M 
C 


7 


7 


d 


7 




o 




yj 










CLASS SIZE 


o 


5 


c 
%f 


t 


3 


1 


$ 








c 










GEN CURRICULUM 


c 


e.5 


1 


2 


6 


7 


6 


SPECIAL EDUC 


o 

O 

£ 


10 




9 


6 


6 


8 


FiNANCiND EDUC 


i 


g 




3 




2 




DISCIPLINE 


x> 
o 
c 


1 


i 

■o 


5 


2 


2 


1 


APATHY 


B 
E 


2 


C 


4* 


5 


4 


3 


FACILITIES 


o 
o 


8 


S 
o 
O 


6 


9 


9 


9 


ADMIN REFORM 




4 




10 


10 


10 


10 


TOTAL NUMBER 
















IN EACH GROUP 


(3)a 


6 


(4)b 


9 


14 


36 


65 



a Number ot city councif members added to county court g^oup 

to make the total of 6. 
b Number ot school board members combined with auperinlendent 

group to maVe the total of 9. 
CoeFNcient ot concordance : .531 

Chi square, 9dl : 19.122 

Probability ol independdnc« : .024 



APPENDIX E 

COMPUTED AVERAGES OF RANKINGS GIVEN 
TEN CRITICAL EDUCATION ISSUES BY 
SIX GROUPS OF TENNESSEANS 
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TABLE A-tO 



COMPUTED AVERAGES OF RANKINGS GIVEN IBN CRITICAL 
EOUCATiON ISSUES BY SIX GROUPS OF TENNESSEANS 





C I Ty 

COUNC 


CNTY 

counr 


SCH 
BRO 


SUPT 


PRINC 


TCHR 


TCHR COMPETENCE 


4.U 


4 67 


4.t1 


451 


4.66 


6.26 


VOCATIONAL EOUC 


4 64 


4.4d 


3 88 


4.40 


5.79 


6.61 


CLASS SI2E 


5 25 


5.90 


5.32 


638 


4.90 


4.24 


GEN CURRICULUM 


6.32 


5,85 


5 93 


5 69 


5 20 


6.03 


SPEC(AL EDUC 


6.46 


667 


6 43 


6.01 


6 22 


6-28 


FINANCING EOUC 


5.04 


5,13 


4 50 


2 85 


3 50 


4.19 


OlSClPLiNE 


4.64 


4,31 


5 n 


6 26 


4.^6 


A.U 


APATHY 


5 04 


493 


S,7S 


5.50 


5.69 


4.66 


FACILITIES 


6 29 


6.43 


6 IS 


538 


6.26 


6.26 


ADMIN REFORM 


7.15 


6 63 


7.79 


aoi 


751 


7.56 


TOTAL NUMBER 
IN EACH GROUP 


29 


67 


113 


107 


104 


317 



Coefficient of concordance : .693 

Chi square, 9df. : 37.43^ 

PfObflbility of itxSepeniSence : .000 



RESULTANT SUMMARY RANKINGS 
(same as Table 3) 





CfTY 
COUNC 


CNTY 
COURT 


SCH 
BRO 


SUPT 


PRINC 


TCHR 


TENNESSEE 
COMPOSITE 


TCHR COMPETENCE 


1 


3 


2 


3 


2 


5 


2 


VOCATIONAL EDUC 


2 5 


2 


1 


2 


7 


6 


3 


CLASS SIZE 


6 


7 


5 


9 


3 


2 


6 


GEN CURRiCULUM 


8 


6 


7 


6 


6 


7 


7 


SPECIAL EDUC 


9 


10 


9 


7 


8 


8 


9 


fiNAt^am EOUC 


4 5 


5 


3 


t 


1 


1 


1 


DISCIPLINE 


25 


1 


4 


e 


4 


3 


4 


APATHY 


45 


4 


6 


S 


6 


4 


5 


FACILITIES 


7 


8 


8 


4 


9 


9 


6 


AOM.IN REFORM 


10 


9 


10 


10 


10 


10 


10 


TOTAL NUMBER 
IN EACH GROUP 


28 


67 


113 


1C7 


104 


317 


73C 



82 

ERIC 



